Comet with Nginx and jQuery

This is an introduction into a basic Comet setup with Nginx and jQuery. You will have to recompile Nginx with NGiNX_HTTP_Push_Module to enable the HTTP Push/Comet functionality. “NHPM” is based on the Basic HTTP Push Relay Protocol and turns Nginx into a very efficient Comet server. This simple recipe will enable you to create live asynchronous web applications utilizing long polling without the complexity of the Bayeux protocol.

First you will need to recompile Nginx with NHPM. If you run into dependency problems please review the Nginx Install Options.

Compiling Nginx with Nginx_HTTP_Push_Module:
NGINX_PUSH_NAME=nginx_http_push_module-0.692
NGINX_NAME=nginx-0.7.67

cd /usr/local/src/archive
wget http://pushmodule.slact.net/downloads/$NGINX_PUSH_NAME.tar.gz
wget http://nginx.org/download/$NGINX_NAME.tar.gz
cd ..
tar zxvf archive/$NGINX_PUSH_NAME.tar.gz
tar zxvf archive/$NGINX_NAME.tar.gz
cd $NGINX_NAME
./configure --prefix=/usr/local/$NGINX_NAME \
--add-module=/usr/local/src/$NGINX_PUSH_NAME
make
sudo make install

Next we will configure a virtual host in Nginx to enable our publisher and subscriber on a single channel called “cheetah”. This configuration allows the queue to hold up to 10 items, and those items will timeout after 5 seconds.

Configuration: (/usr/local/nginx-0.7.67/conf/nginx.conf)
server {
    listen  81;
    server_name cheetah.example.com;

    root /var/www/cheetah.example.com;

    location /cheetah {
        push_channel_group pushmodule_cheetah;
        location /cheetah/pub {
            set $push_channel_id cheetah;
            push_publisher;
            push_message_timeout 5s;        # Give the clients time
            push_message_buffer_length 10;  # to catch up
        }
        location /cheetah/sub {
            set $push_channel_id cheetah;
            push_subscriber;
            send_timeout 3600;
        }
    }
}
Start nginx:
sudo /usr/local/nginx-0.7.67/sbin/nginx

Here is the code for the subscriber that will listen for incoming messages and simply display them. Notice the the use of “If-None-Match” and “If-Modified-Since”. These headers are used to traverse the messages in the queue. The ETAG header is used to uniquely identify messages with the same modification time.

Listen.html: (/var/www/cheetah.example.com/listen.html)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
    <head>
        <title>Listen</title>
        <script type="text/javascript" src="http://www.google.com/jsapi"></script>
        <script type="text/javascript">
        /* <![CDATA[ */
   google.load("jquery", "1.4.2");

   function listen(last_modified, etag) {
       $.ajax({
           'beforeSend': function(xhr) {
               xhr.setRequestHeader("If-None-Match", etag);
               xhr.setRequestHeader("If-Modified-Since", last_modified);
           },
           url: '/cheetah/sub',
           dataType: 'text',
           type: 'get',
           cache: 'false',
           success: function(data, textStatus, xhr) {
               etag = xhr.getResponseHeader('Etag');
               last_modified = xhr.getResponseHeader('Last-Modified');

               div = $('<div class="msg">').text(data);
               info = $('<div class="info">').text('Last-Modified: ' + last_modified + ' | Etag: ' + etag);

               $('#data').prepend(div);
               $('#data').prepend(info);

               /* Start the next long poll. */
               listen(last_modified, etag);
           },
           error: function(xhr, textStatus, errorThrown) {
               $('#data').prepend(textStatus + ' | ' + errorThrown);
           }
       });
   };

   google.setOnLoadCallback(function() {
       /* Start the first long poll. */
       /* setTimeout is required to let the browser know
          the page is finished loading. */

       setTimeout(function() {
           listen('Thu, 1 Jan 1970 00:00:00 GMT', '0');
       }, 500);
   });
        /* ]]> */
        </script>
        <style type="text/css">
            #data {
                margin: .5em;
            }

            #data .info {
                font-weight: bold;
                font-size: 14px;
            }

            #data .msg {
                white-space: pre;
                font-family: courier;
                font-size: 14px;
                margin-bottom: .5em;
                margin-left: .5em;
            }
        </style>
    </head>
    <body>
        <div id="data"></div>
    </body>
</html>

This is the publisher that will put messages into the queue.

Send.html: (/var/www/cheetah.example.com/send.html)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
    <head>
        <title>Send</title>
        <script type="text/javascript" src="http://www.google.com/jsapi"></script>
        <script type="text/javascript">
        /* <![CDATA[ */
    google.load("jquery", "1.4.2");

    function showResult(status, response) {
        $('#result').html('<strong>status:</strong> ' + status +
        '<br /><strong>response:</strong><br />' + response);
    };
 
    google.setOnLoadCallback(function() {
        $('#pub').submit(function() {
            message = $('#message').val();
       
            /* Do not send empty message */
            if (message == '') {
                return false;
            }
       
            $.ajax({
                url: '/cheetah/pub',
                data: message,
                dataType: 'text',
                type: 'post',
                success: function(responseText, textStatus, xhr) {
                    showResult(textStatus, responseText);
                },
                error: function(xhr, textStatus, errorThrown) {
                    showResult(textStatus, errorThrown);
                }
            });
            return false;
        });
    });
        /* ]]> */
        </script>

    </head>
    <body>
        <form id="pub" method="post" action="/cheetah/pub">
            <input type="text" class="message" name="message" id="message" />
            <input class="submit" type="submit" value="send" />
        </form>
        <div id="result"></div></div>
    </body>
</html>
Example Screenshots:

(http://cheetah.example.com:81/send.html) Comet Send Screenshot (http://cheetah.example.com:81/listen.html) Comet Listen Screenshot

Other Methods:

PHP Publish
$ch = curl_init('http://cheetah.example.com:81/cheetah/pub');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "Hello World!");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$ret = curl_exec($ch);
curl_close($ch);
Bash Publish
curl -d 'Hello World' http://cheetah.example.com:81/cheetah/pub

Update:

Here is a working example:
http://cheetah.jamieisaacs.com

18 thoughts on “Comet with Nginx and jQuery

  1. Zeeshi
    #

    hello,

    Nginx is timing out the request after about 10 seconds, even though I have set send_timeout to 7200. I want to develop a real time stats service using nginx_http_push_module. I am using nginx 0.8.53 and nginx_http_push_module 0.692.

    ————————————- Nginx Configuration Begin ——————————

    worker_processes 1;
    events {
    worker_connections 1024;
    }
    http {
    include mime.types;
    default_type application/octet-stream;
    sendfile on;
    send_timeout 7200;
    keepalive_timeout 600;
    push_max_reserved_memory 10M;

    server {
    listen 80;
    server_name localhost;

    location / {
    root html;
    index index.html index.htm;
    }

    ###### Push configuration
    location /publish {
    set $push_channel_id $arg_id; #/?id=239aff3 or somesuch
    push_publisher;
    push_store_messages off; # enable message queueing
    push_message_timeout 2h; # expire buffered messages after 2 hours
    #push_max_message_buffer_length 10; # store 10 messages
    #push_min_message_recipients 0; # minimum recipients before purge
    }
    # public long-polling endpoint
    location /activity {
    push_subscriber;
    set $push_channel_id $arg_id;
    send_timeout 7200; #so that nginx won’t drop connections willy-nilly
    }
    ###### END Push configuration
    error_page 404 /404.html;
    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
    root html;
    }
    }
    }
    ————————————- Nginx Configuration End ——————————

    Following is my test subscriber ruby script:
    ——————————————————————————————————————————-
    require ‘rubygems’
    require ‘em-http’

    def subscribe(opts)

    puts “send request”
    listener = EventMachine::HttpRequest.new(‘http://127.0.0.1/activity?id=’+ opts[:channel]).get :head => opts[:head]
    puts “wait for response”
    listener.callback {
    # print recieved message, re-subscribe to channel with
    # the last-modified header to avoid duplicate messages
    puts “Listener recieved: ” + listener.response + “\n”

    modified = listener.response_header['LAST_MODIFIED']
    subscribe({:channel => opts[:channel], :head => {‘If-Modified-Since’ => modified}})
    }
    listener.errback {
    # print recieved message, re-subscribe to channel with
    # the last-modified header to avoid duplicate messages
    puts “*********** Listener Failed: \n”

    modified = listener.response_header['LAST_MODIFIED']
    subscribe({:channel => opts[:channel], :head => {‘If-Modified-Since’ => modified}})
    }
    end

    EventMachine.run {
    channel = “b6d4e4d8d345f7fb8d850bb676bef0ba”

    # open two listeners (aka broadcast/pubsub distribution)
    subscribe(:channel => channel)
    }
    ——————————————————————————————————————————-

    I added listener.errback block in the above script to catch the timed out requests and resend the polling request. If the publisher does not publish anything on the channel within almost 10 seconds the subscriber request is released by the nginx.

    Following is my test publisher ruby script:
    ——————————————————————————————————————————-
    require ‘rubygems’
    require ‘em-http’

    EventMachine.run {
    channel = “b6d4e4d8d345f7fb8d850bb676bef0ba”

    # Publish new message every 5 seconds
    EM.add_periodic_timer(15) do
    time = Time.now
    publisher = EventMachine::HttpRequest.new(‘http://127.0.0.1/publish?id=’+channel).post :body => “Hello at channel http://www.bumpin.com @ #{time}”
    publisher.callback {
    puts “Published message @ #{time}”
    puts “Response code: ” + publisher.response_header.status.to_s
    puts “Headers: ” + publisher.response_header.inspect
    puts “Body: \n” + publisher.response
    puts “\n”
    }
    end

    }
    ——————————————————————————————————————————-

    Please advise how can I make the nginx hold the request indefinitely for long polling.

    Thanks,
    Zeeshi

  2. thinklinux
    #

    Thanks a lot !!! Just what I needed :)

  3. Jimbobotel
    #

    Thanks! This was exactly what I have been looking for. Saved me many hours.

  4. Gutza
    #

    First off, I must say this is the first sandbox environment that actually works — kudos for that. I suppose people who just want to check it out would appreciate a complete sender/listener environment, as shown below. Caveat: the frameset approach didn’t work in my Firefox 3.x installation, but it worked in IE 8.x and Chrome 9.x; I assume one of my Firefox add-ons interfered somehow, but I never investigated thoroughly.

    Complete nginx/push test

    Having said that, I think you might want to consider adding a note to your post, for completeness: security-wise, this approach is only able to limit the senders (using NHPM’s push_authorized_channels_only directive), but not the listeners; this info is useful for people who want to use Comet for projects which need strict control on data distribution (use case: Google Docs, where any authenticated user can be a sender, but only authorized users can be listeners).

    And finally, on a side note, you might want to consider using separate base names for the domain and the nginx channels; you’re currently using “cheetah” for both (www.cheetah.com, /cheetah/pub, /cheetah/pub and so on). Separating them in a logical fashion (www.example.com, /my_channel/pub) would probably make things easier to understand.

  5. makuchaku
    #

    Exactly what I was looking for!
    Many thanks :)

  6. Coach J Post author
    #

    Adding this line fixes the Firefox Bug:

    listen('Thu, 1 Jan 1970 00:00:00 GMT', '0');

    Here is a working example:
    http://cheetah.jamieisaacs.com

  7. theyouyou
    #

    Is it possible to push to a particular user / ip adress ?

  8. Pingback: Comet and PHP « King Foo

  9. robson
    #

    can i use this to open many DIVs, like facebook wall? its like an web messenger with severous window open, can i use this tecnology to do that?

  10. Pingback: Como trabalhar com Comet Server usando Nginx e JQuery | SWX Softwares - Desenvolvimento e Design Web

  11. mihello
    #

    Thanks,it help me a lot.That’s what I want.

  12. Christopher
    #

    Thanks so much for this! Just what I needed!

    Combine this with the Twitter streaming API and you have a live formattable and embeddable Twitter feed :3

  13. Kevin Hou
    #

    That’s just what I’m looking for. It works great in my Centos server. Thanks very much for your sharing!

  14. Kevin Hou
    #

    By the way, you can use curl to send POST data, and the listener could also get the real-time data.

    curl -X POST -d “great article” http://127.0.0.1/cheetah/pub

  15. Pingback: Simple comet example using php and jquery - PHP Solution - Developers Q & A

  16. Pingback: Comet + Bootstrap | Jackson F. de A. Mafra

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>