SOLUTION: WebSockets, Django Channels, MacOS 10.12, Server 5.2, Apache 2.4.23

Fri 13, January 2017

Finally…

I have WebSocket connections working via Apache in MacOS Server 5.2.

As this caused a lot of head/heartache over the last few weeks I thought I’d better share what I experienced in case someone finds themselves in the same boat!

And the solution is simple.

The short version:

I use MacOS Server 5.2 (ships with Apache 2.4.23) to run a python Django application via the mod_wsgi module.

I had been trying to setup proxypass and wstunnel in MacOS 10.12 & Server 5.2 to handle websocket connections via an ASGI interface server called Daphne running on localhost on port 8001.

I wanted to reverse proxy any WebSocket connection to wss://myapp.local/chat/stream/ to ws://localhost:8001/chat/stream/

From what I had read on all the forums and mailing lists that I had scoured was to simply make some proxypass definitions in the appropriate virtual host and make sure that the mod_proxy and mod_proxy_wstunnel modules were loaded and it would work.

Long story short – from what I understand all of this trouble came down to MacOS Server 5 and one major change:
“A single instance of httpd runs as a reverse proxy, called the Service Proxy, and several additional instances of httpd run behind that proxy to support specific HTTP-based services, including an instance for the Websites service.”

All I needed to do to proxy the websocket connection was the following:

in:
/Library/Server/Web/Config/Proxy/apache_serviceproxy.conf

Add (around line 297 (in the section about user websites, webdav):
ProxyPass / http://localhost:8001/
ProxyPassReverse / http://localhost:8001/

RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule .* ws://localhost:8001%{REQUEST_URI} [P]

I then kicked over the service proxy:

sudo launchctl unload -w /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.apple.serviceproxy.plist
sudo launchctl load -w /Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.apple.serviceproxy.plist

And the web socket connections were instantly working!

The long version:

For many weeks I have been trying to get WebSocket connections functioning with Apache and Andrew Godwin’s Django Channels project in an app I am developing.

Django Channels is “a project to make Django able to handle more than just plain HTTP requests, including WebSockets and HTTP2, as well as the ability to run code after a response has been sent for things like thumbnailing or background calculation.”

My interest in Django Channels came from my requirement of a chat system in my webapp. After watching a several of Andrew’s demos on youtube and having read through the docs and eventually installing Andrew’s demo django channels project I figured I would be able to get this working in production on our MacOS Server.

The current release of MacOS 10.12 and Server 5.2 ships with Apache 2.4.23. This comes with the necessary mod_proxy_wstunnel module to be able to proxy WebSocket connections (ws:// and secure wss://) in Apache and is already loaded in the server config file:

/Library/Server/Web/Config/apache2/httpd_server_app.conf

Daphne is Andrew’s ASGI interface server that supports WebSockets & long-poll HTTP requests. WSGI does not.

With Daphne running on localhost on a port that MacOS isn’t occupying (i went with 8001) the idea was to get Apache to reverse proxy certain requests to Daphne.

Daphne can be run on a specified port (8001 in tis example) like so (-v2 for more feedback):
daphne -p 8001 yourapp -v2

I wanted Daphne to handle only the web socket connections (as I use currently depend on some apache modules for serving media such as mod_xsendfile). In my case the websocket connection was via /chat/stream/ based on Andrew’s demo project.

From what I had read in MacOS Server’s implementation of Apache the idea is to declare these proxypass commands inside the virtual host files of your “sites” in:

/Library/Server/Web/Config/apache2/sites/

In config files such as:
0000_127.0.0.1_34543_.conf

I did also read that any customisation for web apps running on MacOS Server should be made to the plist file for the required web app in:
/Library/Server/Web/Config/apache2/webapps/
In a plist file such as:
com.apple.webapp.wsgi.plist
Anyway…

I edited the 0000_127.0.0.1_34543_.conf file adding:

ProxyPass /chat/stream/ ws://localhost:8001/
ProxyPassReverse /chat/stream/ ws://localhost:8001/

Eager to test out my first web socket chat connection I refreshed the page only to see an error printed in the apache log:

No protocol handler was valid for the URL /chat/stream/. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.

I had read of many people finding a solution at least with Apache on Ubuntu or a custom install on MacOS.
I even tried installing Apache using Brew and when that didn’t work I almost proceeded to install nginx.

After countless hours/days of googling I reached to the Apache mailing list for some help with this error. Yann Ylavic was very generous with his time and offered me various ideas on how to get it going. After trying the following:

SetEnvIf Request_URI ^/chat/stream/ is_websocket
RequestHeader set Upgrade WebSocket env=is_websocket
ProxyPass /chat/stream/ ws://myserver.local:8001/chat/stream/

I noticed that the interface server on port 8001 Daphne was starting to receive ws connections!
However in the client browser it was logging:

“Error during WebSocket handshake: ‘Upgrade’ header is missing”

From what I could see mod_dumpio was logging that the “Connection: Upgrade” and “Upgrade: WebSocket” headers were being sent as part of the web socket handshake:

mod_dumpio: dumpio_in (data-HEAP): HTTP/1.1 101 Switching Protocols\r\nServer: AutobahnPython/0.17.1\r\nUpgrade: WebSocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: 17WYrMeMS8a4ImHpU0gS3/k0+Cg=\r\n\r\n
mod_dumpio.c(164): [client 127.0.0.1:63944] mod_dumpio: dumpio_out
mod_dumpio.c(58): [client 127.0.0.1:63944] mod_dumpio: dumpio_out (data-TRANSIENT): 160 bytes
mod_dumpio.c(100): [client 127.0.0.1:63944] mod_dumpio: dumpio_out (data-TRANSIENT): HTTP/1.1 101 Switching Protocols\r\nServer: AutobahnPython/0.17.1\r\nUpgrade: WebSocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: 17WYrMeMS8a4ImHpU0gS3/k0+Cg=\r\n\r\n

However the client browser showed nothing in the response headers.

I was more stumped than ever.

I explored the client side jQuery framework as well as the the Django channels & autobahn module to see if perhaps something was amiss, and then revised my own app and various combinations of suggestions about Apache and it’s module. But nothing stood out to me.

Then I reread the ReadMe.txt inside the apache2 dir:

/Library/Server/Web/Config/apache2/ReadMe.txt

“Special notes about the web proxy architecture in Server application 5.0:

This version of Server application contains a revised architecture for all HTTP-based services. In previous versions there was a single instance of httpd acting as a reverse proxy for Wiki, Profile, and Calendar/Address services, and also scting as the Websites service.º With this version, there is a major change: A single instance of httpd runs as a reverse proxy, called the Service Proxy, and several additional instances of httpd run behind that proxy to support specific HTTP-based services, including an instance for the Websites service.

Since the httpd instance for the Websites service is now behind a reverse proxy, or Service Proxy, note the following:

It is only the external Service Proxy httpd instance that listens on TCP ports 80 and 443; it proxies HTTP requests and responses to Websites and other HTTP-based services.

I wondered if thus ServiceProxy had something to do with it. I had a look over:

/Library/Server/Web/Config/Proxy/apache_serviceproxy.conf

and noticed a comment – “# The user websites, and webdav”.
I figured it wouldn’t hurt to try adding the proxypass definitions & rewrite rules that people had suggested on the forums as their solution.

ProxyPass / http://localhost:8001/
ProxyPassReverse / http://localhost:8001/

RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule .* ws://localhost:8001%{REQUEST_URI} [P]

Sure enough after restarting the ServiceProxy it all started to work!

http://stackoverflow.com/questions/41287959/mod-proxy-wstunnel-mac-os-x-10-11-6-apache-2-4-18
https://github.com/adamteale/macos-server5-websocket/

Auto mount SMB/AFP/* volumes in Mac OS X

Wed 7, December 2016

Many moons ago I posted some findings on how to get network volumes to auto mount in Mac OS X.

Recently I needed to do this again – but things have changed a little since OS X 10.5 (currently running 10.11).

All the info I found was in this great post:

http://www.gm2dev.com/2015/01/automount-smb-shares-on-osx-yosemite/

Essentially:

edit this file:

/etc/auto_master

add something along the lines of:

/mnt auto_smb -nosuid

(this example creates a mount point on your machine “/mnt” based on the file “auto_smb” that we need to create next.

create a file such as:

/etc/auto_smb

add mount points to this auto_smb file:
(this example creates an smb mount point called “projects” which connects to an IP address of 192.168.1.102 as guest and mounts the  “allthestuff” directory from that machine)

projects -fstype=smbfs,soft ://guest@192.168.1.102/allthestuff

kick it over with:

sudo automount -vc

 

Up until now it has been working well!

 

Learning to be a Djangonaut: Localisation/translation – locale for Chile

Thu 31, December 2015
Djangonaut
A person who is expert in Django web framework.
He is a Djangonaut dude.

In my pursuit to become a gun djangonaut and get my knowledge of the framework sound and stable I have been following along Marina Mele’s fantastic series “TaskBuster Django Tutorial”. It covers a lot of topics that I am keen to understand and be able to explain one day.

My current goal is to have a django web app that has a Restful API that will allow various types of authentication. On top of this I will create an iOS app that can talk to it.

I had started to follow along with a post that Félix Descôteaux had written titled “A Rest API using Django and authentication with OAuth2 AND third parties!”. As I was getting into it I realised that I had better spend some time get a better basic understanding of Django and some other key concepts of python and web app development I had yet to explore. Geez the internet is just so cool when you want to learn something!

Along the way following Marina’s series I hit a bit trouble getting “Localizations” to work – when a site works across multiple languages. Django seems to have a easy to use mechanism but I was having issues with the locale for Chilean Spanish (I always am really…). 🙂

So in an effort to try to give back a little to the wonderful internet for anyone else who may stumble across this post with the same issue…

Basically:

It was a combination of using “es-CL”, “es_CL” & “es-cl” in various places.

 

Property Value
Base locale ID es-CL
Language es
Language Spanish
Country CL
Country Chile

In the base.py (settings file):

LANGUAGES = (
('en', _('English')),
('<strong>es-CL</strong>', _('Spanish (Chile)')),
)

Then in the test_all_users.py I used ‘es-cl‘ wherever necessary.

Then in the tb_test virtual env shell:

$ python manage.py makemessages -l es_CL

I edited the django.po with my translated strings.
(I had to laugh when I read the “po” in django.po for this Chilean translation :] )

Then in the tb_test virtual env shell:

$ python manage.py compilemessages -l es_CL

Now I am passing the tests and seeing the translated page in the browser.

I hope that helps someone out one day!

Chilenismos 1.2

Sat 19, December 2015

About 2 weeks ago I released a reasonable upgrade to Chilenismos just in time for the festive season!

– 380+ Chilenismos!

– 560+ definitions in spanish

370+ definitions in ENGLISH!

– 390+ examples

– “literal translations” in ENGLISH (just for laughs)

– the origen of the Chilenismo (when available)

– App now runs in fullscreen mode

– Optimised for the iPhone 6 display

– swipe support to jump back and forth between the definitions

– little flags!

 

Yes, English definitions! Finally!

 

Chilenismos also now has it’s space on the web!

http://chilenismosapp.cl

 

And a facebook page!

http://facebook.com/chilenismosapp/

 

Grab a copy if you haven’t already.

 

This slideshow requires JavaScript.

Chilenismos – iPhone app v1.0.1

Wed 26, March 2014

Ahhhhhh...

After a much talk over the last 2 years or so it has finally happened, I have made a basic dictionary of Chilean modismos/slang/expressions that is now available for free on the iTunes App Store!

This first version serves as a dictionary of over 170 words and expressions along with definitions and examples. The app also features a quiz game to practice the material along with an easy method of contributing to the more words and suggestions via email.

Check it out on the Chilenismos' page and make sure you grab the link and head on over to the App Store to download it for free.

 

“Play me I’m yours”

Sun 8, December 2013
Play me I'm yours. Pasaba por el metro en Los Leones y escuché a alguien tocando este piano colorido. 
Cool idea! Nice work local government.

#santiago #chile #música
#cosasgratisensantiago
Pasaba por el metro en Los Leones y escuché a alguien tocando este piano colorido. Cool idea! Nice work local government.

latest reel of Magician and friend Adam Mada

Sat 7, December 2013
About 2 months ago I went to sea my friend Adam and Susie and their young son Finn over in Buenos Aires. It was a great weekend. Adam handed me a disk with videos of his work as a magician and asked me to cut him a reel. Adam returned to Australia with his family and I returned to Santiago and we worked together over various screen sharing sessions and produced his latest reel. Here it is:

vfx: cooler projects from the last 2 years here in Chile

Tue 3, December 2013

I've been quite fortunate to have been able to work on some pretty cool projects since freelancing as a Nuke compositor in Santiago about 18 months ago. I realised I haven't shared much on here for too long so here is my start at posting some updates and other coolness.

feels.tv has been completely rebranded and hats off the the great work that they are doing. Whilst checking out the site I came across one of the projects I was involved with earlier this year for AFP Habitat.

Entel Campaña Institucional

This was a cool project working with Feels and the guys at Alaska films. Feels got me onboard to help out with the compositing side of things. We comped up a stadium, integrated cg balloons and extended a crowd amongst other things.

La Ruta del Vino

Not a comp job but a 1 hour mega project in which I was involved as videographer, editor, post producer amongst other things. We did this project at Orangutan and is really something I'm very proud of.

Ladybug Reveal Teaser

This was an idea that Juan Paulo at Leyenda had to help promote the business - there are some very talented dudes working over there in the world of animation. 

Orangutan showreel 2013

Some of the cooler work we did at Orangutan this year.

fcpxImageExporter and my 5 minutes of coding fame

Mon 2, December 2013

A little while ago at Orangutan we had a client ask us to export several hundred frames from a 1 hour video we had produced for them.

We worked in fcpx and manually exporting frames can be quite a task.

So I spent a late afternoon putting together a little app to read fcpxml and use it to export still images based on the markers we had put in the timeline.

I polished it a little more and shared it on the fcp.co forums and over the week the little app had been downloaded over 700 times.

It even got a nice review from Larry Jordan and several retweets from other fcpx pros like Richard Taylor and sites like premiumbeat.com. 

fcpxImageExporter is available over at Orangutan.

fie105