You are not logged in.

Announcement

[2017.01.08] DeployStudio build v1.7.6 (checksum, release note).
[2016.08.26] DeployStudio build v1.6.19 (release note).
[2013.02.23] DeployStudio last universal build v1.5.17 (release note).

#1 2016-12-23 10:47:19

boli
Member
From: Switzerland
Registered: 2016-12-22

DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Hi there

I recently noticed that our DS (1.7.5) workflows hang after starting, they never even do their first step.

At first I thought it was because I updated the server to macOS 10.12.2, but then I found it still works with clients running macOS 10.12.1, it's only the clients running macOS 10.12.2 that hang.

I've since checked privileges of the DS share, reinstalled DS server, re-run the assistant, created new test workflows, both automatic and manual, all to no avail.

More details below, but I'm wondering if others have the same issue, or ideas how to solve?

We run DS Runtime with the regular OS booted on target machines (not on NetBoot/NetRestore/NetInstall images), and the workflows consist of mostly (almost entirely) packages and shell scripts. In the example below it's a single step workflows that should display an alert.

On the client side I see that the DS repository gets mounted at /tmp/DSNetworkRepository:
% ls -l /tmp/
drwx------@  1 root   wheel   568B Dec 22 13:16 DSNetworkRepository/

The logs don't show anything helpful either:
2016-12-22 13:25:42.611 DeployStudio Runtime.bin[34102:1224935] ====================================================================================================
2016-12-22 13:25:42.611 DeployStudio Runtime.bin[34102:1224935] DeployStudio Runtime
2016-12-22 13:25:42.611 DeployStudio Runtime.bin[34102:1224935] DSCore.framework version 1.7.5 (b160915), Copyright 2016 The DeployStudio Team.
2016-12-22 13:25:42.612 DeployStudio Runtime.bin[34102:1224935] MAC address: XXXXX
2016-12-22 13:25:42.623 DeployStudio Runtime.bin[34102:1224935] Network address: ##### (XXXXX)
2016-12-22 13:25:42.634 DeployStudio Runtime.bin[34102:1224935] Network interface speed: AUTOSELECT (1000BASET <FULL-DUPLEX,FLOW-CONTROL,ENERGY-EFFICIENT-ETHERNET>)
2016-12-22 13:25:42.655 DeployStudio Runtime.bin[34102:1224935] Operating System: Mac OS X Version 10.12.2 (Build 16C67)
2016-12-22 13:25:42.655 DeployStudio Runtime.bin[34102:1224935] Date: 16/12/22 13:25:42
2016-12-22 13:25:42.655 DeployStudio Runtime.bin[34102:1224935] ====================================================================================================
2016-12-22 13:25:42.655 DeployStudio Runtime.bin[34102:1224935] 0 plugins were successfully loaded!
2016-12-22 13:25:42.927 DeployStudio Runtime.bin[34102:1224935] Starting Bonjour lookup...
2016-12-22 13:25:44.297 DeployStudio Runtime.bin[34102:1225211] The user 'admin' was successfully authenticated.
2016-12-22 13:25:44.315 DeployStudio Runtime.bin[34102:1224935] Connected to server XXX (1.7.5)
2016-12-22 13:25:44.486 DeployStudio Runtime.bin[34102:1225342] Checking server reachability (server=XXX port=548) ...
2016-12-22 13:25:44.500 DeployStudio Runtime.bin[34102:1225342] Mounting repository share point...
2016-12-22 13:25:44.501 DeployStudio Runtime.bin[34102:1225342] /bin/mkdir /tmp/DSNetworkRepository 2>&1
2016-12-22 13:25:44.511 DeployStudio Runtime.bin[34102:1225342] /bin/chmod 777 /tmp/DSNetworkRepository 2>&1
2016-12-22 13:25:44.521 DeployStudio Runtime.bin[34102:1225342] /bin/chmod 777 /tmp/DSNetworkRepository 2>&1
2016-12-22 13:25:44.531 DeployStudio Runtime.bin[34102:1225342] Mount call...
2016-12-22 13:25:44.725 DeployStudio Runtime.bin[34102:1225342] Parsing result...
2016-12-22 13:25:44.749 DeployStudio Runtime.bin[34102:1225342] Checking repository content...
2016-12-22 13:25:45.131 DeployStudio Runtime.bin[34102:1225342] Checking host firmware version
2016-12-22 13:25:45.304 DeployStudio Runtime.bin[34102:1225342] Macintosh serial number: XXXXX
2016-12-22 13:25:45.304 DeployStudio Runtime.bin[34102:1225342] Macintosh model: MacPro6,1
2016-12-22 13:25:45.304 DeployStudio Runtime.bin[34102:1225342] Boot ROM version: MP61.88Z.0116.B20.1610051057
2016-12-22 13:25:45.304 DeployStudio Runtime.bin[34102:1225342] Firmware ok!
2016-12-22 13:25:48.017 DeployStudio Runtime.bin[34102:1224941] Running workflow: 'bOli Test Alert' (FE101469-930E-451E-9F09-66D9A038C11A)

Last edited by boli (2016-12-23 11:03:36)

Offline

#2 2016-12-23 13:32:11

mjsanders
Member
From: Schiedam, Netherlands
Registered: 2008-09-02
Website

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

The log you show tells me that the runtime starts the workflow 'bOli Test Alert'.
The log (as seen in DS Admin) is usually not updated during running a workflow, just before the client reboots the log is updated.
Take a look the client log (menu window - show logs) (cmd-L)  in Runtime is in real time.

I suspect your workflow encounters an error that is reported locally, but never reported back to the server.

What is the reason you run the runtime locally from a macOS system?
The whole system is designed (and tested) with netbooted clients, so you can expect some workflow steps to behave unexpectedly.
example: when the runtime is ready, it will quit after xx (usually 30) seconds. In netboot this means : reboot. Locally: just quit runtime.

Offline

#3 2016-12-23 13:55:33

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

This _is_ the live/real-time client log. It's identical to the log in the DS Repository/Logs. The runtime does not display the alert, it just hangs. Here's a pic: https://imgur.com/a/c8ATU

/var/log/system.log doesn't show anything related neither on server or client.

As to why run it on a live system: we use it to install the stuff we need, but we keep the OS that is preinstalled from factory because often it's newer than what's available in the app store. For example: the late 2016 MacBook Pros will not boot with a vanilla 10.12.1, because certain drivers are missing. They do boot with 10.12.2 though, that is "unified" once more.

Last edited by boli (2016-12-23 14:07:50)

Offline

#4 2017-01-02 14:17:43

mjsanders
Member
From: Schiedam, Netherlands
Registered: 2008-09-02
Website

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Ok I see your point, but I will repeat that deploystudio is designed to do imaging, with some post-install scripting.
You can do just post-install scripting, but there are a lot of (undocumented) requirements.
Most scripts run best post-poned, execpt if you want to use DS_variables.

Looking at your log, the workflow 'bOli Test Alert' hangs.

What is the workflow?
Can you run the script(s) behind it on a local mac?

Offline

#5 2017-01-03 11:28:30

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

> You can do just post-install scripting, but there are a lot of (undocumented) requirements.

Interesting, as we've been using it to "just" install packages and run scripts for years (and yes we donate occasionally).

> Most scripts run best post-poned, execpt if you want to use DS_variables.

We don't use those.

> Looking at your log, the workflow 'bOli Test Alert' hangs. What is the workflow?

It contains a "display an alert" task. I've never used that kind of task before, but it seemed well suited to reproduce the problem. Any other real workflows hang just the same way.

> Can you run the script(s) behind it on a local mac?

There's no script in this test case, but for our normal workflows, yes I can install the packages and run the scripts manually just fine.

Offline

#6 2017-01-03 13:28:01

mjsanders
Member
From: Schiedam, Netherlands
Registered: 2008-09-02
Website

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Hmm, I am afraid I am out of my ideas.
I use the Runtime also while booted from a local disk (full macOS), so the Runtime.app should work on a normal OS.
I use it to restore the other partition of training Mac's, so workflows with restore and configure tasks only.
Have to admit I never used the 'display an alert' task.

If you run the Runtime locally, the version of OS of the server is hardly important, as long as it supports the DS version.

According to the log the runtime mounts the repository, so that looks good.
To check communication between runtime and deamon: if you leave the runtime at the workflow selector window, is this visible in DeployStudio Admin?
If yes: no idea what to do, but if no: check which server you communicate with, and why it fails.

If you just install software, why not use Munki? (I know, that needs a new setup, and a steep learning curve, but Munki is designed to install software, and you can use the 'Self-Service' strategy too)

Offline

#7 2017-01-03 13:54:28

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

> To check communication between runtime and deamon: if you leave the runtime at the workflow selector window, is this visible in DeployStudio Admin?

Yes, in the DS Admin I see a dark blue icon when the DS Runtime workflow selector is displayed on the client, and the icon turns light blue if the DS Runtime is just quit instead of trying to run a workflow.

> If you just install software, why not use Munki? (I know, that needs a new setup, and a steep learning curve, but Munki is designed to install software, and you can use the 'Self-Service' strategy too)

Thanks for this idea, I've been toying around with the idea of using something else, exactly because it might simplify self service. We're a company of software engineers, so after the initial setup with DS we leave the computer administration mostly to the users themselves. Exceptions to this are rare, for example providing extra workflows to install Office 2016 with all updates, or drivers and definitions for new printers that were physically deployed.

For now our workaround is to run DS first before updating macOS to 10.12.2. I was hoping that maybe by the time new Macs ship with 10.12.2 preinstalled, there will be a fix for this problem.
Though I haven't yet seen any indication that anyone else is having this problem, but it sounds like what I'm doing with DS is weird anyway. ;)

Offline

#8 2017-01-05 00:15:42

admin
Administrator
Registered: 2007-03-29
Website

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Hi, may you try with the nightly build http://www.deploystudio.com/get.php?fp=DeployStudioServer_v1.7.6.dmg ?

It fixes some issues we faced while trying to reproduce your bug.

Offline

#9 2017-01-05 13:40:00

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Hi, yes I'll try it, thank you.

Meanwhile I've also tested a fresh macOS 10.12.2 install in a VM, where DS v1.7.5 does *not* have the problem I described.

I'll check your v1.7.6 on a (real) machine that had this problem now.

Updates:
OK, so I reinstalled macOS 10.12.2 onto another partition on my current machine, installed DS runtime 1.7.5 and had the same problem. So while it does not happen in a 10.12.2 VM, it does happen on another fresh 10.12.2 install on a real machine (Mac Pro late 2013). Also tested on other machines, such as a MacBook Pro or iMac, but those might have OSes that were updated from El Capitan, rather than installed fresh.

Then I installed the 1.7.6 nightly build server and runtime, and still have the same problem. :(

I have yet to try with a fresh DS server install somewhere. The one I'm using has been updated for quite a while. :)

Last edited by boli (2017-01-05 17:56:09)

Offline

#10 2017-01-05 17:58:38

admin
Administrator
Registered: 2007-03-29
Website

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Thanks for the feedback, we'll try to reproduce on a fresh install too.
It looks like a Runtime issue only (probably a deadlock due to concurrent graphics refresh calls).

Offline

#11 2017-01-05 18:11:23

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

I also have another problem, that may or may not be related.

Often I edit workflows with DS Admin running on the DS Server machine (via Apple Remote Desktop), but sometimes I run DS Admin on my machine, and recently I noticed that when I do so it no longer displays task icons whatsoever. When I run DS Admin on the server machine all is well.

Check out the following picture to see what I mean:
http://imgur.com/a/vpCcK

Last edited by boli (2017-01-05 18:13:09)

Offline

#12 2017-01-06 02:14:35

tungates
Member
Registered: 2010-03-25

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Take a look at this http://www.deploystudio.com/Forums/viewtopic.php?id=3922

I had this issue a while ago and pretty sure this solved it

Offline

#13 2017-01-06 10:33:09

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Thanks tungates, that did indeed solve the display problem. For simplicity I used cp -p… instead of tar -p…, that is:
sudo cp -pR /Applications/Utilities/DeployStudio\ Admin.app/Contents/Plugins /Library/Application\ Support/DeployStudio/Plugins

Update: This also solved the "workflow hangs" problem, the workflows now complete once more!

Update 2: In my testing VM (where it just works), the /Library/Application\ Support/DeployStudio/Plugins directory is completely empty. On my real Mac, it doesn't work with an empty Plugins directory. After I emptied it, the workflows stall; and once I copy the Plugins back into it, the workflows run normally.

Removing everything DS and reinstalling just DS Runtime doesn't help (nor did it in my earlier trial of installing just DS Runtime on a fresh macOS 10.12.2 install).

Last edited by boli (2017-01-06 14:49:17)

Offline

#14 2017-01-08 20:29:35

admin
Administrator
Registered: 2007-03-29
Website

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

Are you using the Runtime pkg installer?
By default, the built-in task plugins are located in /Applications/Utilities/DeployStudio\ Runtime.app/Contents/Plugins/.
/Library/Application\ Support/DeployStudio/Plugins should be reserved to custom task plugins.

Offline

#15 2017-01-10 15:22:50

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

> Are you using the Runtime pkg installer?

Yes. And the Plugins are where you said they should be:

% ll /Applications/Utilities/DeployStudio\ Runtime.app/Contents/Plugins
total 0
drwxrwxr-x  26 root  admin   884B Sep 15 00:50 ./
drwxr-xr-x   6 root  admin   272B Sep 15 00:50 ../
drwxrwxr-x   3 root  admin   102B Sep 15 00:04 DSADBindingTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:04 DSAlertTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:04 DSAppleSoftwareUpdateTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:05 DSAutoEnrollmentTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:06 DSConfigureTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:11 DSDiskImageTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:12 DSFileCopyTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:12 DSFirmwareSecurityTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:12 DSFusionDriveTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:13 DSGenericTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:13 DSHostnameTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:14 DSInstallTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:14 DSLicensesTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:15 DSLocalizationTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:15 DSNetworkLocationTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:15 DSODBindingTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:17 DSPartitionTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:17 DSPeriodicNetBootTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:18 DSRestoreTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:18 DSShutdownTask.bundle/
drwxr-xr-x   3 root  admin   102B Sep 15 00:19 DSTargetTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:19 DSTimeTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:20 DSUserTask.bundle/
drwxrwxr-x   3 root  admin   102B Sep 15 00:20 DSWorkflowTask.bundle/

Update:
Oh I see you released 1.7.6 with promising release notes, I'll give it a try!

Update 2:
After installing 1.7.6 Server and Runtime (on different machines) I get the following error when running a workflow:

Workflow aborted
Missing task plugin, please reinstall the Runtime app.
Please check the logs for more details.

Update 3:
In the log it says "0 plugins were successfully loaded!", which sounds suspicious.

Here's the full log:
2017-01-10 15:37:05.688 DeployStudio Runtime.bin[73117:5414456] ====================================================================================================
2017-01-10 15:37:05.688 DeployStudio Runtime.bin[73117:5414456] DeployStudio Runtime
2017-01-10 15:37:05.691 DeployStudio Runtime.bin[73117:5414456] DSCore.framework version 1.7.6 (b170108), Copyright 2017 The DeployStudio Team.
2017-01-10 15:37:05.692 DeployStudio Runtime.bin[73117:5414456] MAC address: aa:ff:ee:aa:ff:ee
2017-01-10 15:37:05.703 DeployStudio Runtime.bin[73117:5414456] Network address: 1.2.3.4 (client.acme.com)
2017-01-10 15:37:05.709 DeployStudio Runtime.bin[73117:5414456] Network interface speed: AUTOSELECT (1000BASET <FULL-DUPLEX,FLOW-CONTROL,ENERGY-EFFICIENT-ETHERNET>)
2017-01-10 15:37:05.731 DeployStudio Runtime.bin[73117:5414456] Operating System: Mac OS X Version 10.12.2 (Build 16C67)
2017-01-10 15:37:05.731 DeployStudio Runtime.bin[73117:5414456] Date: 17/01/10 15:37:05
2017-01-10 15:37:05.731 DeployStudio Runtime.bin[73117:5414456] ====================================================================================================
2017-01-10 15:37:05.732 DeployStudio Runtime.bin[73117:5414456] 0 plugins were successfully loaded!
2017-01-10 15:37:05.927 DeployStudio Runtime.bin[73117:5414456] Starting Bonjour lookup...
2017-01-10 15:37:07.472 DeployStudio Runtime.bin[73117:5414924] The user 'admin' was successfully authenticated.
2017-01-10 15:37:07.489 DeployStudio Runtime.bin[73117:5414456] Connected to server server.acme.com (1.7.6)
2017-01-10 15:37:07.641 DeployStudio Runtime.bin[73117:5414464] Checking server reachability (server=server.acme.com port=548) ...
2017-01-10 15:37:07.647 DeployStudio Runtime.bin[73117:5414464] Mounting repository share point...
2017-01-10 15:37:07.647 DeployStudio Runtime.bin[73117:5414464] /bin/mkdir /tmp/DSNetworkRepository 2>&1
2017-01-10 15:37:07.653 DeployStudio Runtime.bin[73117:5414464] /bin/chmod 777 /tmp/DSNetworkRepository 2>&1
2017-01-10 15:37:07.657 DeployStudio Runtime.bin[73117:5414464] /bin/chmod 777 /tmp/DSNetworkRepository 2>&1
2017-01-10 15:37:07.660 DeployStudio Runtime.bin[73117:5414464] Mount call...
2017-01-10 15:37:07.808 DeployStudio Runtime.bin[73117:5414464] Parsing result...
2017-01-10 15:37:07.818 DeployStudio Runtime.bin[73117:5414464] Checking repository content...
2017-01-10 15:37:07.829 DeployStudio Runtime.bin[73117:5414464] /bin/cp -r "/Applications/Utilities/DeployStudio Runtime.app/Contents/Frameworks/DSCore.framework/Resources"/Scripts/* "/tmp/DSNetworkRepository/Scripts" 2>&1
2017-01-10 15:37:07.863 DeployStudio Runtime.bin[73117:5414464] /bin/chmod -R a+w /tmp/DSNetworkRepository/Scripts 2>&1
2017-01-10 15:37:08.273 DeployStudio Runtime.bin[73117:5414464] Checking host firmware version
2017-01-10 15:37:08.405 DeployStudio Runtime.bin[73117:5414464] Macintosh serial number: X0XXX0X0X0XX
2017-01-10 15:37:08.405 DeployStudio Runtime.bin[73117:5414464] Macintosh model: MacPro6,1
2017-01-10 15:37:08.405 DeployStudio Runtime.bin[73117:5414464] Boot ROM version: MP61.88Z.0116.B21.1610201524
2017-01-10 15:37:08.405 DeployStudio Runtime.bin[73117:5414464] Firmware ok!
2017-01-10 15:37:10.028 DeployStudio Runtime.bin[73117:5414744] Running workflow: 'bOli Test Alert' (FE101469-930E-451E-9F09-66D9A038C11A)

Update 4:
Deleting and reinstalling the 1.7.6 runtime did not help.

FWIW, my user (which I use to launch DS Runtime) does not have admin privileges (never has). Everything in /Applications/Utilities/DeployStudio\ Runtime.app/Contents/Plugins/ is world-readable though. And when I run DS Runtime with an actual admin user (the one I use to install stuff, such as the DS Runtime package), the same thing happens.

Last edited by boli (2017-01-10 15:52:54)

Offline

#16 2017-01-13 14:47:37

boli
Member
From: Switzerland
Registered: 2016-12-22

Re: DeployStudio Runtime hangs after starting workflow in macOS 10.12.2

To reiterate in a shorter post:
With the DSRuntime 10.7.6 on macOS 10.12.2 installations (fresh 10.12.1 install upgraded to 10.12.2, haven't gotten around to creating a 10.12.2 NetInstaller) I reproducibly get the error message

"Workflow aborted
Missing task plugin, please reinstall the Runtime app.
Please check the logs for more details."

when trying to run any DS workflow.

On fresh installs of macOS 10.12.1 it works normally.

---

The workaround of copying Plugins to the Application Support folder still works:
cp -rp /Applications/Utilities/DeployStudio\ Runtime.app/Contents/Plugins/* /Library/Application\ Support/DeployStudio/Plugins/

Last edited by boli (2017-01-13 14:51:10)

Offline

Board footer

Powered by FluxBB