Efficient Realtime Dashboard with Pusher WebHooks
Today we want to talk about a code update we have made on Panda, taking advantage of the Pusher web hooks released a few days ago.
This is a fantastic feature for us as it has considerably reduced the number of api calls made to Pusher, saving us bandwidth and money.
One of the great features of Panda is it’s real time dashboard. Panda can display the new encoding jobs, their progress, and your stats instantly with no effort. Thanks to Pusher, implementing this was really easy.
At such a scale the right design for your software is to separate your app into smaller apps, small apis and workers. In our case, the encoding dashboard is completely separated from the api but they share the same Pusher app. When an event occurs on Panda (like a job progress), the api triggers a Pusher Event and the web app reflect that change immediately.
Before the release, we were facing one problem: we were sending a lot of unnecessary api calls to Pusher.
Panda’s api has no interaction with the user. It only communicates through private and public apis. This means it doesn’t know whether someone is listening to the Pusher channel. We are then pushing thousands of messages even if nobody is looking at the dashboard. The api triggers messages, the web app listens. This is not a two way messaging.
Now with Pusher Web Hooks, our api is able to know when at least one person is looking at the dashboard. As each customer uses a different private channel, that solves our problem.
Our Pusher dashboard is self-explanatory.
Now we would like to show you how we implemented it.
First we need to setup an endpoint for the web hooks. Let’s make a small Sinatra app and use the
Pusher::WebHook object, which comes with the new gem.
class PusherWebHooks < Sinatra::Base post '/pusher' do wh = Pusher::WebHook.new(request) if wh.valid? wh.events.each do |event| case event["name"] when "channel_occupied" channel_occupied(event["channel"]) when "channel_vacated" channel_vacated(event["channel"]) end end else status 401 end return end end
We can setup our notification url inside the Pusher dashboard and we are ready to receive web hooks.
Because splitting your code is the right thing to do, Panda’s encoders are not communicating directly from the api neither. So we somehow need to send this information to them.
To do that, we will use
Redis Sets as we already use Redis to distribute messages across our farm of encoders.
def channel_occupied(channel_name) redis.sadd("pusher:channels", channel_name) end def channel_vacated(channel_name) redis.srem("pusher:channels", channel_name) end
Now our Panda encoders can use this information to check whether the channel we are sending the message to is occupied.
def push_websocket(channel_name, event_name, data) if redis.sismember("pusher:channels", channel_name) Pusher[channel_name].trigger!(event_name, data) true else false end end
Redis commands on sets are O(1) operations which makes the design very efficient.
How simple is that? More information is available inside the Pusher docs. Now you have no excuse not to implement Pusher WebHooks. :)