Justin Rummel . com

Professional Bio/Resume site for Justin Rummel

Casper Suite 9: Cloud and JDS Distribution Points

| Comments

Overview

The Casper Suite has been able to provide installation packages to Managed OS X clients by AFP, SMB, and/or HTTP(s) for a long time, but now JAMF Software has introduced two new methods to provide packages: JAMF Distribution Server (JDS) and Cloud Distribution Point (CDP). Both of these DP installation methods make deploying web based package distribution EXTREMELY easy and quick to stand up in a test or production environment vs. needing to configure multiple services in a Windows or OS X Server setting.

Cloud Distribution Points (CDP)

Cloud Distribution Points are easy to describe as they utilize a Cloud hosting provider to store your DMG or PKG to install on your client machines no matter where they are located. JDS servers are ideal for locations that have security restrictions on port forwarding through a firewall, who don’t want to have non-rack-mountable Mac Mini Server in their DMZ, or a physically diverse workforce where it doesn’t make sense to host installation packages in-house. There are a couple of limitations and requirements for a CDP as follows:

  • You can only have one CDP in your environment. This makes sense as you are trying to get something available outside of your internal network. You need to pick a cloud hosting provider that can support the bandwidth requirements for the number of devices you are trying to support. At this time you have three choices

    • Rackspace
    • Amazon Web Services (S3 and CloudFront)
    • Akamai

All communication between your JSS, your CDP, and your clients will be over HTTPS (port 443) to ensure a proper secure environment.

  • You can only store Packages, in-house iOS apps, and in-house eBooks (no scripts). Scripts can now be stored in the “jamfsoftware” database so you don’t really need to have them as a flat file to download.

  • CDP can be the Master Distribution Point, or you can selectively sync items to your cloud storage.

JAMF Distribution Server (JDS)

A JDS is something very new. From the Admin guide JAMF Software describes a JDS as “instance is a distribution point that is managed by the JAMF Software Server (JSS), similar to a computer or mobile device”. A JDS is a completely separate server that you install on OS X Server (10.6 or greater) or Linux (Ubuntu 10.04 LTS, 12.04LTS, and Red Hat). Some items to note are:

  • JDS can be installed multiple times. In this respect it is like a traditional Distribution Point vs. the one install of a CDP.

  • The first install is your root install. This is important! All additional JDS will be “fed” from the root JDS as the primary source of packages (you can change which server is the ROOT at a later time if you wish).

  • If you have multiple JDS servers installed, you can select the parent/child relationship to help manage which files are synced.

  • This is being done with WebDAV and SSL Certificates. So you want to make sure you know what your are doing with your environment. Either start buying from a third-party vendor (Network Solutions, Verisign, StartSSL) OR make sure you know how to install your internal ROOT CA and Intermediate CA into your JDS Server. 1

A JDS has a complete copy of items to be installed within it’s local repository, therefore it doesn’t make sense to have a traditional Distribution Point a JDS installed on the same server, so pick one: JDS or traditional Distribution Point. You find the file locations of a JDS on JAMF’s kbase Components Installed on JDS Instances.

When moving your scripts and packages to your new JDS, there are some special characters that can’t be used in the file name: / : ? < > \ * | ” [ ]. All scripts are now stored within the jamfsoftware MySQL database vs. a flat “.sh/.py/.perl/.rb” file. There are also a couple of “gotchas” when using the JDS as listed in JAMF’s kbase Migrating Packages and Scripts

- You must use the script editor in the JSS to make changes to the contents of scripts.
- You are no longer able to use scripts in the AppleScript format.
- You are no longer able to deploy non-flat PKGs using Casper Imaging v8.5 or earlier, or Casper Remote v8.x.

Notes


  1. I haven’t got a chance to test out an internal CA yet, but it sounds fun! This may be a future article. 

Sources

  • Casper Admin Guide PDF within the Casper Suite 9 DMG
  • linked JAMF kbase articles

Debug APNS Issues for JAMF Software’s Casper Suite

| Comments

Overview

Hopefully you had time to review my last article on how Apple’s Push Notification System (APNS) works when managing OS X and iOS devices. It’s not required reading to comprehend, but it does provide an an overview on how complicated APNS is AND the beauty in its architecture to make everything happen in an instant! What I now want to discuss is not when APNS works, but when it doesn’t “What are the common things I check when APNS is not working”. Most of these debuging steps are for the initial setup of your APNS environment, if this were working “fine” but now nothing works… there may be something else at hand.

Network APNS issues

First and foremost, when APNS doesn’t work I’m blaming your network. Are you allowing the proper ports out of your environment; specifically ports 2195, 2196, and 5223? You can do some testing for 2195 and 2196 be trying using the ’nc’ command gateway.push.apple.com and feedback.push.apple.com over port 2195 and 2196 respectively, and this needs to happen FROM your MDM (JSS specifically regarding the Casper Suite). Below are successfully examples using the nc command:

nc test for APNS
1
2
3
4
justinrummel@JRummel-MBPr ~> /usr/bin/nc -z -4 -w 10 gateway.push.apple.com 2195
Connection to gateway.push.apple.com 2195 port [tcp/*] succeeded!
justinrummel@JRummel-MBPr ~> /usr/bin/nc -z -4 -w 10 feedback.push.apple.com 2196
Connection to feedback.push.apple.com 2196 port [tcp/*] succeeded!

The second item is your devices connecting to APNS network over 5223. This is the network element that would allow your devices (OS X or iOS), to talk to Apple.

nc test for Push client initialization server
1
2
justinrummel@JRummel-MBPr ~>  /usr/bin/nc -z -4 -w 10 init-p01st.push.apple.com 80
Connection to init-p01st.push.apple.com 80 port [tcp/http] succeeded!

When running these test be sure to perform them their perspective network segments. Most networks are NOT flat except inside your house/home. Server networks may have tighter requirements than WiFi networks for your iPads. Remember, all three of these sessions are being established from the inside to the outside world. Most firewalls are setup to allow this type of network communication as most people are worried about people getting in, not going out of the network environment. Lastly, if your network administrator starts thinking WE HAVE A GIANT DOOR FOR THINGS TO GO OUT ON THREE PORTS! OMG!, request that these ports are only valid for the 17.x.x.x Class A network. Apple owns the whole class A block.

DNS issues

It goes without saying that DNS needs to work from the Server and your clients. I have been in some situations where the ONLY DNS available in the server room is for the internal domain, this way virus cannot find the “command central” and spread their diseased infested “ones and zeros” throughout the servers (this is a lie, but whatever). With no DNS, PUSH commands will never be sent. May need to move your MDM to a DMZ.

Certificate issues

Along with DNS, there may be certificate issues. With Certificates you MUST use the Fully Qualified Domain Name (FQDN). Sorry… I know “server1” is much easier than “server1.domain.tld”, but that doesn’t work when you need certificates to validate. USE the FQDN. In fact, ALWAYS use FQDN in any setting on the JSS with the exception for NetBoot server, and that is a requirement because the bless command is looking for an IP address.

Tokens

Lastly I want to bring up a strange issue that I discovered some time ago when testing OS X 10.8 and the Casper Suite by creating multiple Virtual Machines with VM Ware Fusion. I hope this was an issue because I created 10 VMs on my MacBook Pro Retina, Mid 2012 and started enrolling them into my test JSS. With the help of JAMF Support we discovered at some point my VM’s were not getting a token from Apple. The way I found this was by searching inside the MySQL database. 1

searching for tokens in MySQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
sadmin@auto:~$ sudo mysql -u root -p
[sudo] password for sadmin:
Enter password:

mysql> use jamfsoftware
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select computer_name, computer_id, apn_token from computers;
+---------------------------+-------------+-------------------------------------------------------------------------+
| computer_name             | computer_id | apn_token                                                               |
+---------------------------+-------------+-------------------------------------------------------------------------+
| osxs1                     |           2 | at3cspak ts6mneko ob5gosst sag2poow nug1tyda lee4bluu aro3ogty hep5erbj |
| osxs2                     |           3 | ax8cucai zag8splr te6pijig en0pykvi hy9avkha yi7ergrb nem5splo of5efkis |
| mo                        |           4 | jaz5jori sw7rhtic qu7mesgi mug4paia el6boosu pol8lecd av2pieok sv6ursqg |
| nemo                      |          15 | mi7iaknr tom7idoo log5hyad upl1agrr ol0olsib lea3hitb gyp8tenr sk6lamjo |
| Lion                      |          13 |                                                                         |
| Composer                  |          16 |                                                                         |
+---------------------------+-------------+-------------------------------------------------------------------------+
6 rows in set (0.00 sec)

mysql>

You can see there are two computers that I enrolled into my JSS, however, they never received an APNS token. The end result is these computers will never receive any PUSH commands because Apple has no way of finding the devices. My initial clue that something was wrong was after the two computers were successfully enrolled, I looked for them to assign a Configuration Profile. They were missing in the “Individual Computers” list. It must be a MySQL command that requires the apn_token be return in order to populate the list.

Notes


  1. No these are not my real APNS tokens! You think I would publish that… on the INTERNET!? Randomly generated by [rand]  

Sources

How APNS Works With MDMs That Manage OSX and iOS

| Comments

Overview

Classic Environment

APNS Image 2

  • “Work” location where your MDM (CasperSuite JSS in this example), and other internal servers may exist.
  • “Home” location because people take their equipment home and still do work from the comforts of their couch.

APNS Workflow

This is where the magic happens!

You as the OS X and/or iOS Administrator want your devices to do something. It may be install a Configuration Profile to “lock down”, OR provide a feature such as Email configuration, Wi-Fi Access, or something else… but you want it down NOW! You login to your MDM management page/console, select what devices you want to perform some action. At that point, your MDM does the following:

  • Communicate to Apple’s Push Notification Servers (APNS) over ports 2195, and 2196 to “FIND MY DEVICES”.

APNS Image 3

  • Your devices are already connected to APNS once they turn on, they have Internet connection, AND port 5223 is not blocked. When your device does connect to Apple’s APNS network, it gets a token.

APNS Image 4

  • It’s this token that allows Apple’s APNS network to find and talk to your devices through your firewall. They key element is when APNS push commands are sent, the only bits of information in the payload from Apple is “HEY Device! Talk to your MDM” and nothing else. That is where APNS stops being the middle man and lets a secure communication take over between your devices and MDM only.

APNS Image 5

  • Once the devices received that command, they will then talk to your MDM over their designated port for the next set of commands you wish to execute from the MDM. In my example commands are sent over port 8443 as this is the port for the Casper Suite (but it may be 443 for other MDM’s such as Profile Manager).

APNS Image 6

  • Your devices then do whatever your MDM requests.

Examples

Some examples that the JSS can do to help manage your OSX environment include:

  • Request to get Software Updates from your Internal SUS over port 8088

APNS Image 7

  • Install packages from an SMB (445) or AFP (548) FileShare

APNS Image 8

  • Install packages from an HTTP (80) or HTTPS (443) server. 1

APNS Image 9

  • Force NetBoot, but I’m not going into the port numbers for that as there are too many.

Notes


  1. Be sure your Certificate on your HTTPS distribution point is signed by a third party OR that your internal ROOT CA is already installed on your client machines. 

Sources

Updated XProtectWatch Now With Terminal-notifier

| Comments

XProtectWatch 1.1.0 (now with terminal-notifier support)

I have updated XProtectWatch to now utilize terminal-notifier “a command-line tool to send Mac OS X User Notifications, which are available in Mac OS X 10.8”. To install terminal-notifier is pretty simple, as I have also created a terminal-notifier.sh script that you can run that will download the latest terminal-notifier zip file from github.com, unzip, and move to your /Applications folder (where my script expect terminal-notifier to exist).

rollBack (hack)

In addition to using terminal-notifier, I’ve also created a rollBack.sh script. As noted in XProtectWatch’s README.md, THIS IS A HACK SCRIPT. I’m really just searching for the last two files within the XProtectWatch Shared folder, removing them, then taking the 2nd to last version and copying them back into the system. They will then be updated the next time sudo /usr/libexec/XProtectUpdater runs, which will then trigger the watchProtect.sh script to run. I created mostly for myself to test the terminal-notification function without having to wait for Apple to update their files.

LaunchAgent shuffle

Lastly, if you had downloaded these scripts before, be sure to take note that I had to move the /Library/LaunchDeamon to ~/Library/LaunchAgent in order for terminal-notifier to work. The reason is notification that were triggered by a system process didn’t forward to the current user, therefore, a user’s account had to load the plist file.

what does it take for all these updates? git clone https://github.com/stonyrivertech/XProtectWatch.git

It’s (Log)^2, It’s Big, It’s Heavy, It’s Wood [Mac OSX Server Edition]

| Comments

AFP

Log files

  • /Library/Logs/AppleFileService/AppleFileServiceAccess.log
  • /Library/Logs/AppleFileService/AppleFileServiceError.log

What?! You don’t have these logs? You might need to turn them on as by default they are disabled.

1
sudo serveradmin settings afp:activityLog = yes

This will enable logging for the following items, and rotate the logs on a weekly basis:

  • logLogin
  • logLogout
  • logCreateDir
  • logCreateFile
  • logOpenFork
  • logDelete

DNS

Log files

  • /Library/Logs/named.log

The file contains everything you would want to know regarding your DNS environment, such as reloading of configurations, zones, shutting down, and transferring to any DNS slaves. You will not see these log messages on a slave DNS server, so you must have access to the master. Notice, you will not see WHAT DNS record was created below (it was delete.rummel.co).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
11-Mar-2013 14:52:41.751 reloading configuration succeeded
11-Mar-2013 14:52:41.752 reloading zones succeeded
11-Mar-2013 15:27:02.757 shutting down
11-Mar-2013 15:27:02.757 stopping command channel on 127.0.0.1#54
11-Mar-2013 15:27:02.757 no longer listening on 127.0.0.1#53
11-Mar-2013 15:27:02.757 no longer listening on 192.168.1.121#53
11-Mar-2013 15:27:02.761 exiting
11-Mar-2013 15:27:02.860 zone 0.0.127.in-addr.arpa/IN/com.apple.ServerAdmin.DNS.public: loaded serial 1997022700
11-Mar-2013 15:27:02.861 zone 1.168.192.in-addr.arpa/IN/com.apple.ServerAdmin.DNS.public: loaded serial 2013031101
11-Mar-2013 15:27:02.862 zone rummel.co/IN/com.apple.ServerAdmin.DNS.public: loaded serial 2013031101
11-Mar-2013 15:27:02.862 zone localhost/IN/com.apple.ServerAdmin.DNS.public: loaded serial 42
11-Mar-2013 15:27:02.862 zone justinrummel.net/IN/com.apple.ServerAdmin.DNS.public: loaded serial 2013031101
11-Mar-2013 15:27:02.862 managed-keys-zone ./IN/com.apple.ServerAdmin.DNS.public: loaded serial 0
11-Mar-2013 15:27:02.863 running
11-Mar-2013 15:27:02.863 zone 1.168.192.in-addr.arpa/IN/com.apple.ServerAdmin.DNS.public: sending notifies (serial 2013031101)
11-Mar-2013 15:27:02.863 zone rummel.co/IN/com.apple.ServerAdmin.DNS.public: sending notifies (serial 2013031101)
11-Mar-2013 15:27:02.863 zone justinrummel.net/IN/com.apple.ServerAdmin.DNS.public: sending notifies (serial 2013031101)
11-Mar-2013 15:27:02.881 client 192.168.1.122#51602: view com.apple.ServerAdmin.DNS.public: transfer of '1.168.192.in-addr.arpa/IN': AXFR-style IXFR started
11-Mar-2013 15:27:02.881 client 192.168.1.122#51602: view com.apple.ServerAdmin.DNS.public: transfer of '1.168.192.in-addr.arpa/IN': AXFR-style IXFR ended
11-Mar-2013 15:27:03.369 client 192.168.1.122#63695: view com.apple.ServerAdmin.DNS.public: received notify for zone '1.168.192.in-addr.arpa'
11-Mar-2013 15:27:03.373 client 192.168.1.122#51603: view com.apple.ServerAdmin.DNS.public: transfer of 'rummel.co/IN': AXFR-style IXFR started
11-Mar-2013 15:27:03.374 client 192.168.1.122#51603: view com.apple.ServerAdmin.DNS.public: transfer of 'rummel.co/IN': AXFR-style IXFR ended
11-Mar-2013 15:27:03.374 client 192.168.1.122#51604: view com.apple.ServerAdmin.DNS.public: transfer of 'justinrummel.net/IN': AXFR-style IXFR started
11-Mar-2013 15:27:03.374 client 192.168.1.122#51604: view com.apple.ServerAdmin.DNS.public: transfer of 'justinrummel.net/IN': AXFR-style IXFR ended
11-Mar-2013 15:27:03.375 client 192.168.1.122#64866: view com.apple.ServerAdmin.DNS.public: received notify for zone 'rummel.co'
11-Mar-2013 15:27:03.875 client 192.168.1.122#56747: view com.apple.ServerAdmin.DNS.public: received notify for zone 'justinrummel.net'

Installer

Log files

  • /var/log/install.log

Did you just double click a “.pkg” file (or something that looks like yellow junk in a box)? The actions of that installation are recorded here. You could also install items by using the installer command (a scripting FYI).

Open Directory

Log files

  • /Library/Logs/slapconfig.log

    Setting up and configuring Open Directory services

  • /Library/Logs/PasswordService/ApplePasswordServer.Service.log

    Successes and failures authenticating with local network users

  • /Library/Logs/PasswordService/ApplePasswordServer.Error.log

    Errors in the Password Service

  • /var/log/slapd.log

    LDAP services

  • /var/log/opendirectoryd.log

    Core Open Directory functionality

There is so much going on in terms of Open Directory it’s hard to begin, as it deals with:

  1. LDAP (OpenLDAP specifically)
  2. Password (Password Service)
  3. Kerberos (now Heimdal vs. MIT Kerberos)

You can read about Open Directory on Apple’s man page opnedirectoryd, or reference the error codes in Apple’s developer section. There are some key troubleshooting steps I usually perform when trying to debug authentication issues:

  1. ping -c 4 ODM.IP.address (it should work)
  2. host ODM.IP.address (it should report back your server’s fully qualified domain name)
  3. host ODM.domain.tld (it should report back only one IP address)
  4. open afp://ODM.domain.tld (you should get an authentication prompt)
  5. dscl /Search -read /Users/*username* NFSHomeDirectory (should return “/Network/Servers/server.domain.tld/Volumes/path/to/username”)
  6. dscl /Search -read /Users/*username* HomeDirectory (should return “<home_dir><url>afp://server.domain.tld/Users</url><path>username</path></home_dir>”)
  7. ntpq -p; ntpdc -c loopinfo (do this on the client and server to verify NTP settings are using the same server)
  8. kinit *username*@ODM.SERVER.TLD (this should be your Kerberos realm so the FQDN is in all caps)
  9. klist -a (Verify TGT and Auth time)

Remember, you can always increase the log level of Open Directory by following Apple’s kbase article and issue:

1
2
3
4
5
# debug
odutil set log debug

# return to default
odutil set log default

* The logging level will persist through restarts.
* Other logging levels are also available: “alert”, “critical”, “error”, “warning”, “notice”, and “info”.
* For more information please refer to the manual pages for the odutil utility (such as “man odutil”).
* Generally, debug logging should only be used to troubleshoot Open Directory service-related issues because debug logging can generate large amounts of log messages. If you need more detailed information about Open Directory events but do not wish to use “debug”, consider using “info” instead.

System

Log files

  • /var/log/system.log

Apple decided to stop utilizing the security.log file for ‘interesting’ items, and now just creates noise in system.log. Grep for the following items:

  • grep sudo /var/log/system.log

    Anyone using the ‘sudo’ command for elevated privileges

  • grep backup /var/log/system.log

    Time Machine ‘backup’ for the server to a secondary drive (not the Time Machine backup service)

  • grep kernel /var/log/system.log

    Kernel messages (such as sandboxd lookup errors)

  • grep bootp /var/log/system.log

    NetBoot and DHCP notices

  • grep kdc /var/log/system.log

    Kerberos log messages for individuals who authenticate for services

Web Services

Log files

General Apache Info

  • /var/log/apache2/access_log

    You may see these two files symlinked in Console.app under /Library/Logs/ => “WebServer”

  • /var/log/apache2/error_log

WebDAV

  • /Library/Logs/AppleFileService/AppleFileServerError.log
  • /Library/Logs/WebDAVSharing.log

CalDAV

  • /var/log/caldavd/access.log

Profile Manager

  • /Library/Logs/ProfileManager/devicemgrd.log
  • /Library/Logs/ProfileManager/profilemanager.log
  • /Library/Logs/PostgreSQL/PostgreSQL.log
  • /Library/Logs/PostgreSQL/PostgreSQL_server_Services.log

    If you have ever had to “redo” some of your work and Postgress, you needed to sudo serveradmin start postgres_server to get things done. There is a log for that.

VPN

Log files

  • /var/log/ppp/vpnd.log

Who just logged into your network? Look for “authorized for access”:

1
grep "authorized for access" /var/log/ppp/vpnd.log

Sources