Option `worker_on_master` is being ignored

[jaley]

I think recent changes have broken this option.

cluster.yaml file:

docker_repo: <private repo image>
size: 10
size_low_pri: 50
subnet_id: <private subnet id>
username: spark
vm_size: standard_e32_v3
worker_on_master: false

If I ssh to the master instance, I see the following:

$ sudo docker exec -it <container id> /bin/bash

$ env | grep AZTK 
AZTK_WORKING_DIR=/mnt/batch/tasks/startup/wd
AZTK_MIXED_MODE=True
AZTK_PYTHON_VERSION=3.5.4
AZTK_IS_WORKER=true
AZTK_WORKER_ON_MASTER=False
AZTK_MASTER_IP=10.128.80.91
AZTK_IS_MASTER=true

Note that AZTK_IS_MASTER and AZTK_IS_WORKER are both true despite AZTK_WORKER_ON_MASTER=False.

I believe this is because the install.py script is using the string value "False" in a boolean context here. The expression bool('False') evaluates to True in Python.

[paselem]

@jaley thanks for pointing it out. We'll have a fix out today.

[paselem]

@timotheeguerin - I think we can fix this by doing

is_worker = not is_master or os.environ["AZTK_WORKER_ON_MASTER"] == "False"

This is not ideal though since we seem to be mixing "False", "false", "True" and "true" throughout the code. We can probably patch this case, but I think we need to standarize

[jaley]

Thanks for the fast turnaround guys, will pull it in and test in our overnight ETL jobs.