Android, Development Tools, Work

Android AOSP Build – Selective Repo Sync for faster synchronization

Lately, I wrote and customize a lot of android modules on AOSP – and somehow run lot of repo sync — all AOSP builder should know by now that it is super slow (even though I use custom beefy instance on GCP to compile my AOSP code)

Googling for solution e.g. StackOverflow only give recommendation to to a shallow clone by adding --depth 1 to repo init to make sure we shallow clone the repo and also to run sync as below

repo sync  -f --force-sync --no-clone-bundle --no-tags -j$(nproc --all)

But it still took around 1-hour to completely finish.

One Friggin Hour, specially on 98-99% part of the project

Now this may be obvious for some experienced AOSP builder that we can run selective sync on specific projects such as below:

repo sync {project-name} {project2-name}

Note that the project-name is the one listed on your manifest entry <project name="x".

But it’s dawning on me (as super inexperienced AOSP customizer) that we can also run selective sync for new project that we just added on main manifest or local manifests.

For example – I just add new project to my local manifests

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project path="common/drivers/amlogic" name="l9_amlogic_driver" remote="lumio" revision="master" groups="v11" />
</manifest>

Commit and push the local-manifest . Pull it on the compiling instance

cd .repo/local_manifests/
git pull origin main
cd ../..
repo sync l9_amlogic_driver
> remote: Total 0 (delta 0), reused 0 (delta 0)
> Fetching: 100% (1/1), done in 1.144s
> Garbage collecting: 100% (1/1), done in 0.004s
> repo sync has finished successfully.

So there you go, repo sync for selective project sync (new project or updated project)

Android, Debug, Troubleshooting

OrangePI PC Android display on DVI monitor

So just got one of those OrangePI PC from ALI Express, it’s one of `rPi` compatible board that have Android 4.4 OS on it. (the other alternatives that I know are BananaPi and ODroid).  It’s based on Allwinner H3 processor and most of the OS are based on sunxi kernel

The first thing I do was scourging trough the forum and download Android 4.4 image base for it.

To install the Android image, AllWinner provide tool called `PhoenixCard` that is only available for Windows OS. (Android image is not `dd` able images – so you cannot use `dd` or `piBaker`)

Let me warn you that PhoenixCard is a weird beast (there’s a bug that cause it to intermittently fail) – and it need some compatible of card-reader hardware. So if you experience a lot of failures on burning images to microSD – try other card-reader hardwares (I try 3 types of card reader – a Sony card reader, mac card reader (using VM) and internal generic ASUS laptop card reader – the ASUS one works fine)

Plug the OPIPC to HDMI monitor – it works fine.

The thing is you cannot plug it to DVI monitor using HDMI-to-DVI converter because the HDCP protection is enabled by default – so to plug it to DVI monitor you will need to disable HDCP protection on it. There’s a lot of guide to fix it on the official forum – but most of the time it’s only for Linux based OS.

For Android you will need to go the extra miles.

  1. Download the `linux sdk source code` for your hardware here  (see section Orange PI PC and find the one with linux sdk source code title). It’s a huge file – so prepare some Coffee.
  2. Extract it into case-sensitive file system (I use VM with ubuntu 14.04 on my Mac, ext4 is case sensitive, mac osx default one is not)
  3. You will need to download and compile sunxi-tools. We will need fex2bin to convert sys_config.fex to it’s compiled binary version
  4. Go to the extracted folder. You will find brandy, buildroot, linux-3.4, out and tools folder
  5. Run `cd tools/pack/chips/sun8iw7p1/configs/dolphin-p1` and edit `sys_config.fex` in it. You will need to find section named `[hdmi_para]` and add these lines
    [hdmi_para]
    hdmi_used = 1
    hdmi_power = "vcc-hdmi-18"
    
    ; Adding DVI monitor compability by disabling HDCP and enabling CTS-compat
    hdcp_enable = 0
    hdmi_cts_compatibility = 1
  6. Go to `tools/pack` folder and run
    cp -fv chips/sun8iw7p1/configs/dolphin-p1/sys_config.fex out/sys_config.fex > /dev/null 2>&1
    cp -fv chips/sun8iw7p1/bin/boot0_sdcard_sun8iw7p1.bin out/boot0_sdcard.fex > /dev/null 2>&1
    unix2dos out/sys_config.fex > /dev/null 2>&1
    pctools/linux/mod_update/script out/sys_config.fex > /dev/null 2>&1
    fex2bin out/sys_config.fex out/sys_config.bin > /dev/null 2>&1
  7. Patch boot0 using generated sys_config.bin
    pctools/linux/mod_update/update_boot0 out/boot0_sdcard.fex out/sys_config.bin SDMMC_CARD > /dev/null 2>&1
  8. Copy Compiled Uboot and patch using generated sys_config.bin
    cp -fv chips/sun8iw7p1/bin/u-boot-sun8iw7p1.bin out/u-boot.fex > /dev/null 2>&1
    pctools/linux/mod_update/update_uboot out/u-boot.fex out/sys_config.bin > /dev/null 2>&1
  9. Write to SDCARD (Change /dev/sdX to use your own sdcard path, it’s /dev/sdf on mine)
    cd out
    sudo dd if=boot0_sdcard.fex of=/dev/sdf bs=1k seek=8 > /dev/null 2>&1
    sudo dd if=u-boot.fex of=/dev/sdf bs=1k seek=16400 > /dev/null 2>&1

After all of that your OrangePi PC Android should be working on DVI monitor.

Devops, Troubleshooting, Ubuntu

Cleanup source from .git artifacts (tronsmart android source)

Was spending these couple of days to compile `Tronsmart’s Draco H3 Android` . The SDK that they provide are 15 Gigabytes in size >_<

Not really sure why they just archive their work folder like that, 15 gigs in itself is a zip full of compiled binaries, few android compiled images, broken symbolic links, nested `.git` folders and hosted on mediafire to download. (Now.. compiling this will be a long journey in itself :P)

So anyhow I clean `.git` artifacts recursively for some rooms to breathe. (besides the `.git` is not usable since it’s point to their internal development server)

To list all files, grep ones with `.git` in the path and then delete it

 find . | grep \\.git | xargs rm -rf 

Keep in mind that grep is using regex so the expression above will be matched with anything that have `.git` in it (nested .git folder, all files inside that git folder, .gitignore, .gitkeep, whatever.git.whatever etc.)

If you are not really sure and want to check what files that are going to be removed. Just use `echo` to test it, It will list all files that matched the expression

 find . | grep \\.git | xargs echo 

 

Arduino, Uncategorized

Hydroponics (Aeroponic) Arduino Timer

Been reading about Hydroponics and Aeroponik system for a while and I figure that it can be my new hobby that I can implement on my home backyard and since I also been tinkering with Arduino for a while (mostly basic stuff) I figure to combine these into automated one.

The Tower Setup

It’s actually variant of Hydroponics (Aeroponic) since the water is sprayed on the root on some specific intervals. My father and brother build a vertical one using PVC pipe. The schema of the tower is basically based on this guide

The ingredients are:

  • PVC pipe
  • Portable Stove (for heating PVC pipe)
  • Old paint bucket for the reservoir and base
  • Old mineral water plastic cups
  • Drill
  • Pipe
  • Pump

IMG_20151115_221254

They basically drill the appropriate hole size. heat the pipe using portable stove and bent the hole so it shaped like the picture above.

IMG_20151115_221303

We use old paint bucket as the water reservoir and base. Put one flexible hose inside the pipe for the pump to deliver the water to the top of the tower and let it drip trough the provided holes on top of the tower.

The Timer Setup

Most tutorial will recommend timer to turn on for 10 minutes and turned off for 10 minutes – the point is do not let roots of the plant not watered for more than 25 minutes. For my setup I program the timer to be 10 minutes on and 15 minutes off.

The ingredients:

  • Arduino (Buy 10 of them from AliExpress here)
  • Relay Board (make sure to look for 5v. I buy it from Tokopedia here)
  • A cheap lock n lock compatible container for water proofing
  • Some jumper wires
  • A cheap 5v charger (use old Nokia charger)
  • A breadboard
  • Double tape
  • Epoxy or Silicon Glue Sealing

First prepare the container. I hacksaw the breadboard for it to be fit inside the small container

IMG_20151115_220604

There’s a lot of tutorial out there about Arduino + Relay schematics. I use this one on Instructables as a reference

IMG_20151115_191436

Don’t forget to recheck the wiring since it will involve mains. (Mains can kill you!!)

To power the Arduino, I rip open cheap ass and old Nokia charger.

I put the Nokia charger on the top lid and stick it with glue so it will get separated with the arduino board. Make sure there’s no contact between both boards since this charger board is mostly run in mains.

IMG_20151115_214957

Make sure the relay have output wires for connection with pumps

IMG_20151115_220103

Seal holes with epoxy or silicon glue to make sure it’s tight and ready for outdoor usage.

IMG_20151115_215053

I program the Arduino timer using slightly modified `blink` code. As you can see it’s just simple delay and everything is hard coded in it 🙂

/*
* This code is for aerophonic timer
*/

unsigned long OFF_TIME = 15;
unsigned long ON_TIME = 10;

void setup() {
// init D13 and D10 pin
pinMode(11, OUTPUT);
pinMode(10, OUTPUT);

// turn on for ON_TIME
digitalWrite(11, LOW);
digitalWrite(10, LOW);
}

void delayMinute(unsigned long delayInMinutes) {
delay(delayInMinutes * 60 * 1000);
}

void loop() {
// turn on for ON_TIME
digitalWrite(11, LOW);
digitalWrite(10, LOW);
delayMinute(ON_TIME);

// turn off for OFF_TIME
digitalWrite(11, HIGH);
digitalWrite(10, HIGH);
delayMinute(OFF_TIME);
}

And here’s it in action next to the tower.

IMG_20151116_090513

Development Tools, Mac, Troubleshooting

Arduino Clone USB Driver on OSX (CH340, CH341)

So, most cheap Arduino I can found in Indonesia’s marketplace is basically a SMD version that based on CH340 USB Serial chip.

You can find the drivers on this site here http://www.wch.cn/download/CH341SER_MAC_ZIP.html, but there’s easier way to install the driver if you already have brew installed. (OSX developer must have `brew` installed! MUST HAVE! :P)

First Install cask (OSX developer who have brew also must have cask :D).

Run the cask recipe


brew cask install caskroom/homebrew-cask/wch-ch34x-usb-serial-driver

(it’s basically a script that is doing the driver downloading and installing for you)

Open up your Arduino and make sure there’s new `tty.wch` device listed on the open port.

Additional notes:

If somehow it’s still not detected, try to restart and change the cable. The UNO that I receive only have USB with Power (not include data)

Debug, IOS, OSX, Troubleshooting

Enable Remote Debugging for Safari Mobile iOS (iPhone and iPad of course)

Just realized that safari on mobile can also support remote debugging like chrome + adb.

So one day I was having issue with quirky CSS bug that only occurs on Safari Mobile, so the first thing I do is turning on the iOS emulator.

If you want to turn on iOS emulator without turning on Xcode; You can run it using Terminal with the command below:

open /Applications/Xcode.app/Contents/Developer/Applications/iOS\ Simulator.app

Screen Shot 2015-09-11 at 11.19.29 PM

You emulate any kind of iOS devices by toggling the device on Hardware menu.

Of course I turn on Mobile Safari and try to reproduce the quirky CSS bug on it. Well Yup… it’s reproducible on the Emulator (so we can safely assume that this is the same Mobile Safari on real devices) and the next logical step is to enable Developer options.

Go to Emulator `Settings` -> `Safari` -> `Advanced` and turn on the `Web Inspector` option

Screen Shot 2015-09-11 at 11.25.44 PM

After that open Mobile Safari and point it to your web page.

Open Desktop Safari on your Mac. And make sure that you have a `Develop` menu shown on top

Screen Shot 2015-09-11 at 11.28.02 PM

If you don’t have it – go to `Safari Preferences` -> `Advanced` and make sure the `Show Develop menu in menu bar` is checked

Screen Shot 2015-09-11 at 11.29.05 PM

Go to `Develop` menu – and your Emulator should be listed. If it’s not – try to restart your Desktop Safari (and make sure you Mobile Safari is active and showing a web page)

Screen Shot 2015-09-11 at 11.30.33 PM

Click on the appropriate page. And a developer tool will show up.

Screen Shot 2015-09-11 at 11.33.41 PM

Happy debugging!

Devops, Ubuntu

Install Imagemagick with WEBP on Ubuntu

Somehow standard imagemagick on Ubuntu does compiled with `WEBP` support, There’s a bug report with “wishlist” level on Ubuntu about it here

So we need to compile it on our own, this is what I do on my Ubuntu machine

First: When you are planning to compile things on your own – you need to install the build-essential

sudo apt-get install build-essential

And then install plethora of dependencies that may be needed by imagemagick

sudo apt-get build-dep imagemagick -y

Download latest imagick and extract it

wget http://www.imagemagick.org/download/ImageMagick.tar.gz
tar xf ImageMagick.tar.gz 

Download latest libwebp and extract it

wget http://downloads.webmproject.org/releases/webp/libwebp-0.4.3.tar.gz
tar xf libwebp-0.4.3.tar.gz

Download dependencies for libwebp

sudo apt-get install libjpeg-dev libpng-dev libtiff-dev libgif-dev

Compile libwebp

cd libwebp-0.4.3
./configure --prefix=/usr
make
sudo make install

Compile imagemagick! (note: you may want to change the version number to the latest imagemagick)

cd ImageMagick-6.9.2-0/
./configure --prefix=/usr
make
sudo make install

Aaaand done!

Troubleshooting, Work

Mass Updating iPads (plus installing supervision mode)

Today in the office, we receive a lot of iPads from the client. These iPads are going to be used as kiosk devices. The thing is those iPads are still using iOS 7.0 (and we are developing in iOS 8.0 to utilize latest API) so we will need to update all of this iPads.

There’s few scenarios that we can do:

  1. Connect to internet and let these iPads update – in which super slow and will makes all my colleagues kill me due to abusing the internet connection 😛
  2. Setup a proxy server to cache iPad firmware update – and do number #1. Ain’t got time to do this 😛
  3. Download firmware from Apple developer, connect it to iTunes and plug and update each iPad manually – one by one
  4. Download Apple Configurator, download firmware, and plug 4 USB cables (since it’s the current max USB port on iMac – does not have USB hub on hand)

Of course given the situation the fastest way is the option number 4.

This is what I do:

  1. Download Apple Configurator here https://itunes.apple.com/en/app/apple-configurator/id434433123?mt=12
  2. If you have already download the latest firmware (and don’t want the Apple Configurator to redownload firmwares) you can put the downloaded firmwares to /Users/YOURUSER/Library/Containers/com.apple.configurator/Data/Library/Caches/ if there’s no Firmware folder – create one yourself and put *.ipsw files there. Screen Shot 2014-10-09 at 4.27.30 PM
  3. Setup the configuration, create a Profile (to connect it to specific Wifi, preload it with custom application, adding MDM link and make sure it to skip some steps in setup assistant) Screen Shot 2014-10-09 at 4.21.38 PM Screen Shot 2014-10-09 at 4.23.52 PM
  4. Click on Prepare button
  5. Plug as many iPad as possible
  6. Let it update – when it finish (give you the Halo message) unplug and replace it to other iPad, repeat this until all of your iPad is updated and provisioned with the new profile.

For basic MDM, we temporarily use https://meraki.cisco.com/ (until we establish our own MDM infrastructure).

It’s basically remote management and inventory tool for devices (so you can setup policies remotely), push Enterprise applications to be installed and manage many things related to iDevices.

Nginx, Security, Troubleshooting

Installing HTTPS/SSL certificate on Nginx

OK, this writing is specific to Godaddy certificate since that is what I’ve been doing recently – but the steps below can be applied to any Certificate Provider

You need to have request CSR signature of the Domain. This is basically to generate a private key that is unique to you (your domain). If you already have one then reuse that! if you don’t have any request CRS – create one CSR By Using this link https://www.digicert.com/easy-csr/openssl.htm

The link basically help you in generating terminal command similar to the one below (WARNING: do not use the command below 😛 in case you didn’t notice on the CN= part, it’s for my domain. Use the link I told you above)

openssl req -new -newkey rsa:2048 -sha256 -nodes -out star_mydomain_co_id.csr -keyout star_mydomain_co_id.key -subj '/C=ID/ST=Jakarta Barat/L=Jakarta/O=My Organization/OU=Web Administration/CN=*.mydomain.co.id'

After executing the command line, you’ll get *.csr and *.key file. The .csr file is Certificate Signing Request – you will need to provide this file to your SSL provider later. The .key file is your private key that is unique to you – of course you MAY NOT spread these files.

Keep these files – we will need it for later. Now use the generated Certificate Signing Request / *.csr to register for SSL in Godaddy (or your provider), they will ask you to upload your certificate and then they will provide you a signed Certificate File .crt. Depend on the provider – they will also provide you with intermediate/bundle certificate.

For example GoDaddy give me 2 files:

the-signed-domain-certificate.crt
gd_bundle.csr

Deploy these files to your machine – In Nginx the provider certificate need to be combined. I combine it using

cat the-signed-domain-certificate.crt gd_bundle.crt > star.mydomain.intermediate.2014.01.01.crt

Is a good practive to append expiration date to the certificate name. Save these files to local path in your server. We usually use /opt/key/. Remember to change the permission of the files to 600 (root readable)

Now go to your nginx sites-available config and add

server {
    listen          443;
    server_name     mydomain.co.id;

    ssl on;
    ssl_certificate         /opt/key/star.mydomain.intermediate.2014.01.01.crt
    ssl_certificate_key  /opt/key/star_mydomain_co_id.key

Restart your nginx. And test out your HTTPS.

Update:
Since google is trying to kill legacy SSL signature that is using SHA1, here’s a good article that will help you to avoid “Connection not Trusted” on future Google Chrome versions

notprivateSo, if you use Godaddy – at first they will give you SHA1 intermediate certificate – but then you can Re-Key your certificate to have SHA2 signed one. Here’s Godaddy explanation about re-key

Development Tools, Nodejs, PHP, Troubleshooting

Bash Script (!#) Shebang Issue (Vagrant, Linux, Windows)

Now, just found this issue yesterday. This happen on Vagrant that have Linux guest machine and using Windows as host machine.

Our shebang script suddenly give us

: No such file or directory

In one of the developer machine. But it works normally on the others. So I try to take a look at it.

The error is happening on one of the machine that is using Windows as the host machine and it does not happening on Linux / Mac’s. So it must be Windows thing right?

Judging from the editor – I cannot see any differences from the shebang lines below.


#!/usr/bin/env php
#!/usr/bin/env node

But suddenly I remember that Windows use CRLF and Linux use LF as line endings.

Never heard of such case in shebang script, I try to prove it by creating a simple hello world that is using proper LF line ending and it works properly.

So yeah if you got : No such file or directory error on your shebang script and you use Vagrant and have multi-OS developers – check your line-ending 🙂

The cause:

  1. Using Vagrant
  2. Using Linux as Guest Machine
  3. Using Windows
  4. Have some shebang scripts
  5. Does not properly set .gitattributes to properly resolve line endings

The fix:

  1. Properly set .gitattributes to properly force line ending – we use text eol=lf (see Github Article about dealing with Line Endings)
  2. Temporary fix to use dos2unix tool to convert the file line-ending