Developing a CRUD grid with IgniteUI, Node.js, Express, Mongoose, MongoDB and Nginx

Alexander Todorov / Wednesday, May 8, 2013

In this blog post I am going to create a fully-functional JavaScript & HTML application from scratch, which supports RESTful CRUD operations. I am going to use the out of the box IgniteUI grid functionality for handling the creates, updates, deletes, and the retrieval of records.

I’ve tried to create a simple architecture which, at the same time, satisfies the following conditions:

High performance
Production-ready - should be able to run an app in production, using the same components, with minimal changes to the configs. The topology and the core of the architecture and frameworks of choice should be unchanged. Should be able to scale easily and be hosted in environments such as Amazon’s EC2 cloud
Use of JavaScript on both the client, and the server. One language to rule them all
Use of popular open source frameworks (such as Node, express, etc.). While we do have a lot of samples, helpers, and content related to ASP.NET MVC, we haven’t spent a whole lot of time showing you the beauty of our toolset when used in the context of other platforms
No custom code should be necessary for the CRUD RESTful logic on the client. This should be all handled by the IgniteUI grid, as out of the box functionality. Since the 12.2 IgniteUI release, the grid component supports RESTful create/read/update/delete operations completely out of the box.

The software stack I’ve chosen consists of the following components:

IgniteUI – powering our presentation layer. I’ve used the latest 13.1 internal build, you can basically use any build from 12.2 and above

Nginx – it’s one of the most popular and high-performance http servers, and I am going to use it for two reasons:

a) To serve static content – images, css, js, and html pages

b) For load balancing Node.js processes

Nginx is used by organizations like Netflix, GitHub, Wordpress and is often preferred over Apache. I’ve used the Nginx-1.2.3 for Windows, even though you can pick any of the latest stable versions for your OS of choice

Node.js – one of the most popular server-side JavaScript frameworks. Has many modules developed by the community - ORM stacks, web servers, etc. Companies like Yahoo and LinkedIn are making heavy use of Node, and it has proven to be a very scalable and high-performance choice for building web apps in particular. And of course, what’s really sweet is the fact that you actually code your server-side logic in JavaScript! I’ve used v0.8.22 of Node, but you can go to http://nodejs.org and grab the latest current binary (which is v0.10.5 at this time).

Express.js – Express is a web framework built on top of Node.js. What we are going to use it for is implementing the GET, POST, PUT, and DELETE request handlers for our backend part. As you will see from the code, it’s really great in the sense that you only need a couple of lines of code to implement something like that. The version of Express.js that I am using is 3.0.0.

Mongoose – you are going to be “Wow!” when you see what this ORM framework can do. You can literally model and persist your documents and objects in a matter of minutes. Using mongoose to load and store objects, and bind them to the IgniteUI grid via Express is actually so easy, that initially I did not really believe my own eyes. I am having Mongoose 3.6.9 installed.

MongoDB – One of the most popular document-oriented NoSQL storage solutions. One of the best features of mongo which I particularly like is the fact that all documents are encoded as BSON (Binary JSON), which makes processing and retrieval of collections extremely efficient and natural. Considering the Ignite UI grid is so JSON-friendly, this makes it a perfect choice for a web app. Also, clustering (sharding: http://docs.mongodb.org/manual/sharding/) is very easy to setup with Mongo.

The following diagram depicts the architecture of the application:

Note that starting more than one Node process is not mandatory, but the setup makes it quite seamless. Nginx will basically take care of the load balancing for you.

We are going to setup forwarding in our Nginx conf file, so that we can start multiple instances of our node.js web app, in order to support load balancing. Our RESTful API is going to be implemented in the Node app, using Express, Mongoose and MongoDB, and our HTML pages, including all other resources are going to be served by Nginx.

Let’s get down to business. First you will need to install Nginx and Mongo, and fire them up. MongoDB may complain that it’s missing some folders when you just start its mongod executable, but you can create those under , and then give it some arguments:

mongod –dbpath data\db

Before starting the nginx executable, let’s change the nginx conf file, so that we setup forwarding to our node processes. Open conf\nginx.conf (depending on the OS you’re using, that may be different, please refer to the nginx docs), and add the following in the “http” section – I’ve marked the new lines in bold:

gzip on;

# can be used for load balancing

upstream igniteui_upstream {

server localhost:8082;

}

server {

listen 8081;

#charset koi8-r;

#access_log logs/host.access.log main;

location / {

proxy_redirect off;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Forwarded-Proto $scheme;

proxy_set_header Host $http_host;

proxy_set_header X-NginX-Proxy true;

proxy_set_header Connection "";

proxy_http_version 1.1;

proxy_pass http://igniteui_upstream;

}

location ~ ^/(.+\.html|images/|img/|javascript/|js/|css/|stylesheets/|flash/|media/|static/|robots.txt|humans.txt|favicon.ico) {

root html;

access_log off;

expires max;

}

What all of the above does, is:

It redirects all requests to the node process, except requests which are for resources that are html pages, that are located in images/ js/ css/ , and so on.
It ensures we can add multiple entries in the upstream declaration, if we would like to start multiple node processes for our app.
It sets the port on which our Nginx will listen to, that is, 8081
Our node.js process’ web server will listen to 8082, and nginx will forward the :8081 requests to :8082 (except the ones we’ve specified in the second “location” declaration).
Our CSS, JavaScript, images and HTML resources will reside in the “html” folder of our Nginx installation, while we can start our server.js node application from anywhere in the file system.

For your reference, I have attached my nginx.conf.

Next, we will need to install Node.js, Express and Mongoose. After installing Node, the easiest way to install Express and Mongoose is using npm. Enter the following commands in a cmd prompt with admin privileges:

npm install express –g

npm install mongoose –g

Then open a text editor, enter the following code and save it as server.js:

/* 1. setup database */
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/products');

var Schema = mongoose.Schema;
var ProductSchema = new Schema({
    name: {type: String, required: true, unique: true},
    unitprice: {type: Number, required: true},
    description: {type: String, trim: true},
    discontinued: {type: Boolean}
});
    
var Product = mongoose.model('Product', ProductSchema);
/* 2. setup the express framework, you can install by: npm install express */
var express = require('express');
var app = express();
/* from the docs: * Request body parsing middleware supporting JSON, urlencoded,

and multipart requests. * This middleware is simply a wrapper the json(),

urlencoded(), and multipart() middleware. */
app.use(express.bodyParser());
/* simulate DELETE and PUT methods */
app.use(express.methodOverride());

/* REST CRUD API */
/* Retrieve all products */
app.get('/', function (req, res) {
    Product.find({}, function(error, data) {
        res.json(data);
    });
});
/* Get Product by ID */
app.get('/:id', function (req, res) {
    Product.find({_id: req.params.id}, function (error, data) {
        res.json(data);
    });
});
/* Create new Products */
app.post('/new', function (req, res) {
    var product_data = {
        name: req.body.name,
        unitprice: req.body.unitprice,
        description: req.body.description,
        discontinued: req.body.discontinued
    };
    var product = new Product(product_data);
    product.save(function (error, data) {
        if (error) {
            res.json(error);
        } else {
            console.log("Added New Product");
            res.statusCode = 201;
            res.send();
        }
    });
});
/* Update an existing Product by ID */
app.put('/update/:id', function (req, res) {
    Product.update(
        {_id:req.params.id},
        {
            $set: {
                name: req.body.name,
                description: req.body.description,
                unitprice: req.body.unitprice,
                description: req.body.description,
                discontinued: req.body.discontinued
            }
        },
        false,
        true
    );
    console.log("Updated Existing Product with ID: " + req.params.id);
    res.send();        
});
/* Delete Product by ID */
app.delete('/delete/:id', function (req, res) {
    return Product.findById(req.params.id, function (error, product) {
        return product.remove(function (error) {
            if (error) {
                console.log(error);
            } else {
                console.log("deleted product: " + req.params.id);
                return res.send();
            }
        });
    });
});
app.listen(8082);
console.log("Server running at localhost:8082");

And run it by calling node server.js in a command prompt. If you are on Windows, and you get errors that Node cannot find express or mongoose, please set your NODE_PATH System variable (System Properties -> Advanced -> Environmental Variables ->

NODE_PATH => C:\Program Files (x86)\nodejs\node_modules

And the code above is really all we need in order to setup CRUD on the server. It’s less than 100 lines of code, isn’t that simply amazing? The beauty of it is that it’s compliant with the RFC specs for GET, POST, DELETE and PUT, so you can use it with any client-side technology that implements the same requests, according to the specs. And the IgniteUI grid and data source components do all of that. Let me go through the code in sections, so that you can get a better understanding of what we are doing:

- First we are connecting to MongoDB using Mongoose’s API, and defining a schema for a Product entity, which has a couple of properties:

/* 1. setup database */
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/products');

var Schema = mongoose.Schema;
var ProductSchema = new Schema({
    name: {type: String, required: true, unique: true},
    unitprice: {type: Number, required: true},
    description: {type: String, trim: true},
    discontinued: {type: Boolean}
});
    
var Product = mongoose.model('Product', ProductSchema);

Note that you don’t need to do anything specific to Mongo, such as issuing commands to create documents. Mongoose handles all of that for you.

Then we are setting up Express:

/* 2. setup the express framework, you can install by: npm install express */
var express = require('express');
var app = express();
/* from the docs: * Request body parsing middleware supporting JSON, urlencoded,

and multipart requests. * This middleware is simply a wrapper the json(),

urlencoded(), and multipart() middleware. */
app.use(express.bodyParser());
/* simulate DELETE and PUT methods */
app.use(express.methodOverride());

And finally, we are implementing our RESTful API. We need to implement several functions – getting all Products, creating a new product, deleting an existing product, and updating a product. Let’s go through them one by one:

app.get('/', function (req, res) {
    Product.find({}, function(error, data) {
        res.json(data);
    });
});

Here we are retrieving all products, and feeding the results as JSON to the response. The way Express and Mongoose bind together is really beautiful.

Retrieving a specific product:

/* Get Product by ID */
app.get('/:id', function (req, res) {
    Product.find({_id: req.params.id}, function (error, data) {
        res.json(data);
    });
});

Note how we are specifying the URL format, this will allow us to process requests in the form of http://hostname/someID. The req.params.id part (Express.js) gives you the “someID” part. For more information on this, please check this out:

http://expressjs.com/api.html#app.param

We code our UPDATE and DELETE request handlers in a similar way:

/* Update an existing Product by ID */
app.put('/update/:id', function (req, res) {
    Product.update(
        {_id:req.params.id},
        {
            $set: {
                name: req.body.name,
                description: req.body.description,
                unitprice: req.body.unitprice,
                description: req.body.description,
                discontinued: req.body.discontinued
            }
        },
        false,
        true
    );
    console.log("Updated Existing Product with ID: " + req.params.id);
    res.send();        
});
/* Delete Product by ID */
app.delete('/delete/:id', function (req, res) {
    return Product.findById(req.params.id, function (error, product) {
        return product.remove(function (error) {
            if (error) {
                console.log(error);
            } else {
                console.log("deleted product: " + req.params.id);
                return res.send();
            }
        });
    });
});

Note that since we’re sending JSON, it’s very easy to get individual properties by using the req.body.propName syntax.

We have to also implement our CREATE handler, it’s very important to set the response status code to 201, so that it conforms to the RFC:

/* Create new Products */
app.post('/new', function (req, res) {
    var product_data = {
        name: req.body.name,
        unitprice: req.body.unitprice,
        description: req.body.description,
        discontinued: req.body.discontinued
    };
    var product = new Product(product_data);
    product.save(function (error, data) {
        if (error) {
            res.json(error);
        } else {
            console.log("Added New Product");
            res.statusCode = 201;
            res.send();
        }
    });
});

Finally, we start our node server with this line of code:

app.listen(8082);

Now, after we have installed mongoDB, Nginx, and have started them, and have started our node.js process for the server (server.js), we can setup our HTML page which will visualize the data in the grid and allow us to do CRUD on the Product objects.

Go to the html folder of your Nginx installation, and place the js and css folders from your IgniteUI installation there. Then create a crud.html file. I am attaching all of that in the blog post, so don’t worry if you are not sure about something.

You can check out the full contents of the crud.html from the app.zip, attachment, but basically this is the main part, our grid definition, the rest is just referencing javascript and CSS resources:

<body>
    <script type="text/javascript">
        $(function () {
            $("#grid1").igGrid({
                dataSource: "/",
                columns: [
                    {key: "_id", headerText: "Product ID"},
                    {key: "name", headerText: "Product Name", dataType: "string"},
                    {key: "unitprice", headerText: "Unit Price", dataType: "number"},
                    {key: "description", headerText: "Description", dataType: "string"},
                    {key: "discontinued", headerText: "Discontinued", dataType: "bool", format: "checkbox"}
                ],
                autoGenerateColumns: false,
                primaryKey: "_id",
                restSettings: {
                    create: {
                        url: "/new"
                    },
                    update: {
                        url: "/update"
                    },
                    remove: {
                        url: "/delete"
                    }
                },
                features: [
                    {
                        name: "Updating", 
                        columnSettings: [
                            {columnKey: "_id", readOnly: true}
                        ]
                    }
                ],
                height: "400px"
            });
            $("#grid1").bind("iggridupdatingeditrowended iggridupdatingrowdeleted", function (e, args) {
                args.owner.grid.saveChanges();
            });
            /* $("#save").click(function () { $("#grid1").igGrid("saveChanges"); }); $("#rollback").click(function () { $("#grid1").igGrid("rollbackChanges"); }); */
        });
    </script>
    
    <br/>
    <table id="grid1"></table>
</body>

We first import all the necessary CSS and script references, on which IgniteUI depends – jQuery, jQuery UI. After that we import the IgniteUI specific scripts that we need. The configuration of the grid is pretty simple. We can use autoGenerateColumns (true by default), but since our Product objects will contain some properties we don’t want to show, and are for internal use, we’d like to define our own columns collection. In this way we will also make the column headers text better. We don’t have to include the “_id” property (which is autogenerated for us, as soon as a new Product gets created), but I will do it anyway – if you use a tool like Firebug, you can check out the actual request/response structure, when we are manipulating records.

And now comes the cool part – notice that in order to enable CRUD – all the way from the client , to the server, DB, and back, we only need to enable Updating, and define our URLs for every CRUD action – as simple as that !

                restSettings: {
                    create: {
                        url: "/new"
                    },
                    update: {
                        url: "/update"
                    },
                    remove: {
                        url: "/delete"
                    }
                },
                features: [
                    {
                        name: "Updating", 
                        columnSettings: [
                            {columnKey: "_id", readOnly: true}
                        ]
                    }
                ]

We also want to make the _id column non-editable.

Since the IgniteUI grid and its datasource components can support editing on two levels – client, and server - we have to propagate our changes to the server by calling saveChanges on every edit, that’s why we need the following code in order to persist to the server:

$("#grid1").bind("iggridupdatingeditrowended iggridupdatingrowdeleted", function (e, args) {

    args.owner.grid.saveChanges();
 });

In this way, as soon as you create a record, update it, or delete it, you will get a request immediately sent. I have commented some code below the above part, which will also enable you to accumulate edits, and commit them by clicking on a “Save Changes” button, you can also experiment with that, if you wish to.

Note that when you enable autoCommit in the grid (which also enables it in the data source component behind the scenes), your changes will be persisted to the client store, but they won’t be automatically propagated to the server. You’ll also get two arrays with transactions – the local pending transactions, and a global list of transactions – this includes all of the transactions which have been committed locally, but haven’t been propagated to the backend. You can find more info on this topic by following this link:

http://es.infragistics.com/help/topic/88cc9763-ce79-4ed8-8da3-c34a61e5a96d

Now, when you enter the following URL in the browser:

http://127.0.0.1:8081/crud.html

(Please ensure that you have Nginx, MongoDB, and the node process running :

>node server.js

)

You will see the following nice grid:

Now you can add, update, and delete records using the Grid’s updating UI. You can track the requests with Firebug (Firefox) or Dev tools (Chrome):