Web Applications using MongoDB – MongoDB is one of the best tools for serious web developers, and this article on MongoDB will look at everything you need to know to build web applications with MongoDB. But before we begin, let’s take a closer look at the popular tool.

Image source: pixabay.com

What is MongoDB?

In the simplest terms, MongoDB is a cross-platform document-oriented NoSQL database that uses JSON-like documents using dynamic schemas, called BSON documents, instead of following the conventional relational database (RDB) structure.

That’s a lot of terms that you may not be familiar with, so let’s break them down first.

Now, a document-oriented database functions on the basic concept that a data document contains ‘metadata’, which are values used to identify the type of data and make it easier to store and retrieve when needed. For instance, let’s look at the following two lines of data:

John Hancock
Architect

A look at these two lines should give you an idea of what this data refers to: a person’s name and occupation. A machine, however, cannot make out this distinction. It needs to be told the type of data.

In an RDB, this type can be stored in a key-value store that’s separate from the document itself. However, with a document-oriented database, you get to determine the type of data as you type the code. So you end up writing a bit like this:

Popular Course in this category

MongoDB Training Program (4 Courses, 2 Projects)4 Online Courses | 2 Hands-on Projects | 22+ Hours | Verifiable Certificate of Completion | Lifetime Access
4.5 (4,179 ratings)

Course Price

View Course

<name> John Hancock </name>
<occupation> Architect </occupation>

And there you go; now the type of data has been defined right within the document. This gives a flexibility that is not present in RDBs, which is why document-oriented databases is so popular for web applications.

After all, web applications are subject to regular changes and updates. Of course, RDBs have their advantages too, and document-oriented databases are not perfect for every situation, but let’s not digress.

Let’s move onto JSON, BSON and schemas. JSON refers to JavaScript Object Notation, but it is actually an open standard data format that can work with languages other than JavaScript.

A schema is the structure of the database, defined as tables in case of RDBs. Since document-oriented databases do not use such tables, they are essentially structure-less, or schemaless.

As for BSON, well, that just stands for Binary JSON, and it’s the data storage format for the MongoDB database. That’s all you need to know about it at this stage.

Now, with the basics out the way, let’s look at why MongoDB is a good choice for making web applications:

Why go for MongoDB?

There are several reasons for the popularity of MongoDB for building web applications. MongoDB is not a perfect tool; far from it, in fact. MongoDB has several challenges, and it is not the right fit for all web applications. But MongoDB works well when it comes to web applications for the following reasons:

MongoDB is schemaless

The MongoDB database can hold collections of documents with each document having its own size, content, and number of fields. You can go back and change the key values, or add and delete them as you want to.

This also makes the structure of a single object clear and straightforward, and MongoDB is also easy to scale out when needed.

MongoDB is general-purpose

The database can be used for a number of web applications, including customer data management, content management, data hubs, mobile apps, Internet of Things, Big Data, product and asset catalogs and database-as-a-service.

MongoDB supports auto-sharding

Relational databases generally scale vertically, in which a single server hosts the complete database. This can limit the scale of operation, with the solution being to add horizontally and add servers rather than concentrating capacity on a single server.

Sharing of a database is done to avoid this and make hardware act as a single server. Developers have to deploy multiple relational databases across machines, and develop application code for distributing the data, queries and aggregate the results.

MongoDB and other NoSQL databases support auto-sharing, which means that they automatically spread out data across various servers without the application being made aware of the composition.

MongoDB is especially useful with cloud computing. As a developer, you don’t have to build expensive and complex platforms for supporting web applications, but can focus on writing the actual application code.

A few more terms to learn about MongoDB

Alright, so we have now covered the basics and benefits of MongoDB. Before we get our hand dirty, let’s look at a few more terms that you should be familiar with:

Database: a database contains collections, and each database has its own files on the file system. A typical MongoDB server will have several databases.
Collection: a group of MongoDB documents is called a collection, and this is the equivalent of a table in RDBs. A single database will have a collection, and each document will have different fields. Unlike tablets, collections do not have a schema.
Document: A document is a bunch of key-value pairs and has a dynamic schema, which means that different documents in the same collection can have different field sets and structures.

Building web applications

The web applications being built for this particular MongoDB guide is an RSS aggregator like Google Reader. The web applications will have two key components: A REST API and a Feed grabber.

To understand what a REST API is, you need to first know a little about the MEAN stack. The code for a feed grabber, can be found even on the github directory.

Understanding the MEAN stack

The MEAN stack stands for the following:

MongoDB, which is the database to be used for the web application
js, which is a light web application framework
js, which is a framework for creating JavaScript and HTML5 rich web applications
js, which is a server-side JavaScript interpreter

For the web applications being discussed here, a REST API will be created instead of an HTML user interface. The Representational State Transfer (REST) API is a lighter-weight alternative to XML- based APIs and can serve as the base for any interface type, despite not having a user interface.

REST functions on a client-server model, with the client sending HTTP commands to the HTTP server, along with variable parameters encoded in the URL describing the target object of the action. The server then replies with the JSON and result code.

Now, MongDB and JavaScript both react well with JSON, making the MEAN stack especially suitable for the web applications.

CRUD

The acronym CRUD stands for create, read, update and delete. These are common database operations that translate in HTTP as post, get, put and delete.

Building a REST API

Let’s begin by creating a REST API, developing the foundation for building the web applications. The application could be web-based or designed for Android, iOS or other specific platforms. To begin with, let’s list down the things that the app will be able to do:

Allow users to create an account
Enable them to subscribe or unsubscribe to feeds
Give them the ability to read and mark feed entries as read or unread
Track RSS feeds for monitoring
Store user information in their respective accounts
Track user feed subscriptions
Track a feed entry already read by a user

Data modeling

Based on the requirements, the web applications will need four collections for managing the information:

User collection
Feed collection
User-feed entry mapping collection
Feed entry collection

Here is a closer look at each one:

Feed collection

The following JSON is a good example for modeling a feed collection:

{ "_id, ObjectId("523b1153a2aa6a3233a91412")

"requiresAuthentication": false,

"permanentlyRemoved": false,

"feedURL": "http://feeds.reuters.com/reuters/topNews",

"title": "Reuters",

"bozoBitSet": false,

"enabled": true,

"etag": "4bL78iLSZud2iXd/vd10mYC32BE",

"link": "http://www.reuters.com/ ",

"permanentRedirectURL": null,

"description": "Reuters news feed” }

The JSON document above is an RSS feed for Reuters, tracking information about the latest news published on the website. There are a lot of fields that you see here, but the most important ones for the web applications to function is the URL of the feed and the feed description.

The URL is important for fetching content from the right source, and the description is important because it provides a summary for the feed.

The rest of the fields, meanwhile, are for internal use. You could add more or remove some of the as you feel like. Right at the top of the document is the _id field. Every document needs to have this field, and the ID must be unique.

In the above example, an ID has been created manually for the field. If you miss this out, MongoDB will automatically make one.

Tracking and collecting feed entries

If you want to track more news feeds, you can add more of them using the same format above. Once you are done adding the feeds, it is time to track and collect them. Given below is a document for feed entry collection:

{ "_id": ObjectId("523b1153a2aa6a3233a91412"),

"description": "Martin Shkreli, the poster boy for prescription drug price increases who was arrested for securities fraud this week, has stepped down from his post as chief executive officer of Turing Pharmaceuticals Inc, the company said on Friday”,
"title": "Turing Pharma names chairman to replace Shkreli as CEO",

"summary": "Turing Pharma names chairman to replace Shkreli as CEO”,
"content": [{ "base": "http://www.reuters.com/",

"type": "text/html",

"value": ”LOTS OF HTML HERE",

"language": "en" }], "published Date": ISODate("2015-12-18T18:13:58+0000"),

"link": " http://www.reuters.com/article/us-usa-crime-shkreli-ceo-idUSKBN0U122R20151218",
"feedID": ObjectId("523b1153a2aa6a3233a913f8") }

Once again, you can see the _id field filled in, along with some other fields such as the description, summary and title. The content field uses an array, and MongoDB allows arrays to store documents. This sub-document arrangement can be particularly useful in situations where all the information has to be held together.

Take a note of the feedID; it has the same ObjectId type and the value of the _id as the earlier feed entry document for the Reuters feed. This offers a referential model, so you can query the feed collection on the same ID value to return the Reuters document.

Tracking the user

Here is an example document for keeping track of users:

{ "_id" : ObjectId("54ad6c3ae764de42070b27b1"),

"active" : true,

"email" : "johnhancock@example.com",

"firstName" : "John",

"lastName" : "Hancock",

"sp_api_key_id" : "6YQB0A8VXM0X8RVDPPLRHBI7J",

"sp_api_key_secret" : "veBw/YFx56Dl0bbiVEpvbjF”,

"lastLogin" : ISODate("2015-01-07T17:26:18.996Z"),

"created" : ISODate("2015-01-07T17:26:18.995Z"),

"subs" : [ ObjectId("523b1153a2aa6a3233a913f8"),

ObjectId("54b563c3a50a190b50f4d63b") ], }

Here, the user has three key attributes: first name, last name and email address. There are also two elements that can be used with the Stormpath user management API: sp_api_key_secret and sp_api_key_id.

The final field, subs, is the subscription array and tells the feed that the user is subscribed to.

Feed entry mapping

The final collection her enables the mapping of users to feeds and tracking the feeds that have been read:

{ "_id" : ObjectId("523b2fcc054b1b8c579bdb82"),

"read" : true,

"user_id" : ObjectId("54ad6c3ae764de42070b27b1"),

"feed_entry_id" : ObjectId("523b1153a2aa6a3233a91412"),

"feed_id" : ObjectId("523b1153a2aa6a3233a913f8") }

As you can see, a basic true/false Boolean is used for marking whether the feed has been read or not. Now, the user should be able perform the following functions:

Creating an account
Subscribing or unsubscribing to feeds
Reading feed entries
Marking feeds or entries as read or unread
Reset the password

Here is a list of how to map these operations to HTTP routes and commands.

Route	Command	Description	Variables
/user/enroll	POST	Register a new user	firstName lastName email password
/user/resetPassword	PUT	Password Reset	email
/feeds	GET	Get feed subscriptions for each user with description and unread count
/feeds/subscribe	PUT	Subscribe to a new feed	feedURL
/feeds/entries	GET	Get all entries for feeds the user is subscribed to
/feeds/<feedid>/entries	GET	Get all entries for a specific feed
/feeds/<feedid>	PUT	Mark all entries for a specific feed as read or unread	read = <true \| false>
/feeds/<feedid>/entries/<entryid>	PUT	Mark a specific entry as either read or unread	read = <true \| false>
/feeds/<feedid>	DELETE	Unsubscribe from this particular feed

Secure HTTP is the standard approach used for sending sensitive details like passwords in a production environment

Next steps: real world authentication

User authentication is important for securely managing users, passwords and resets. There are several ways to authenticate user accounts.

For instance, you could use a Passport Plugin with Node.js, which is useful to authenticate with Facebook, Twitter and social media accounts.

The above example depends on the use of Stormpath, a REST API that supports authorization and authentication via API keys and offers user management-as-a-service.