Introduction to NoSQL

Background

Many applications rely on databases to store data, such as passwords, email addresses, or comments. The most popular database engines are relational (e.g. Oracle and MySQL). However, over the past decade, non-relational databases, also known as NoSQL databases, have become increasingly more common, with MongoDB now being the 5th most used database engine (as of November 2022).

There are four main types of NoSQL databases, and unlike relational databases, which all store data similarly in tables, rows, and columns, the way NoSQL databases store data varies significantly across the different categories and implementations.

In this module, we will focus solely on MongoDB, as it is the most popular NoSQL database.

Introduction to MongoDB

MongoDB is a document-oriented database, which means data is stored in collections of documents composed of fields and values. In MongoDB, these documents are encoded in BSON (Binary JSON). An example of a document that may be stored in a MongoDB database is:

{
  _id: ObjectId("63651456d18bf6c01b8eeae9"),
  type: 'Granny Smith',
  price: 0.65
}

Here we can see the document's fields (type, price) and their respective values ('Granny Smith', '0.65'). The field _id is reserved by MongoDB to act as a document's primary key, and it must be unique throughout the entire collection.

Connecting to MongoDB

We can use mongosh to interact with a MongoDB database from the command line by passing the connection string. Note that 27017/tcp is the default port for MongoDB.

[!bash!]$ mongosh mongodb://127.0.0.1:27017

Current Mongosh Log ID: 636510136bfa115e590dae03
Connecting to:          mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.0
Using MongoDB:          6.0.2
Using Mongosh:          1.6.0

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

test>

We can check which databases exist like this:

test> show databases
admin       72.00 KiB
config     108.00 KiB
local       40.00 KiB

Creating a Database

MongoDB does not create a database until you first store data in that database. We can "switch" to a new database called academy by using the use command:

test> use academy
switched to db academy
academy>

We can list all collections in a database with show collections.

Inserting Data

Similarly to creating a database, MongoDB only creates a collection when you first insert a document into that collection. We can insert data into a collection in several ways.

We can insert a single document into the apples collection like this:

academy> db.apples.insertOne({type: "Granny Smith", price: 0.65})
{
  acknowledged: true,
  insertedId: ObjectId("63651456d18bf6c01b8eeae9")
}

And we can insert multiple documents into the apples collection like this:

academy> db.apples.insertMany([{type: "Golden Delicious", price: 0.79}, {type: "Pink Lady", price: 0.90}])
{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId("6365147cd18bf6c01b8eeaea"),
    '1': ObjectId("6365147cd18bf6c01b8eeaeb")
  }
}

Selecting Data

Let's say we wanted to check the price of Granny Smith apples. One way to do this is by specifying a document with fields and values we want to match:

academy> db.apples.find({type: "Granny Smith"})
{
  _id: ObjectId("63651456d18bf6c01b8eeae9"),
  type: 'Granny Smith',
  price: 0.65
}

Or perhaps we wanted to list all documents in the collection. We can do this by passing an empty document (since it is a subset of all documents):

academy> db.apples.find({})
[
  {
    _id: ObjectId("63651456d18bf6c01b8eeae9"),
    type: 'Granny Smith',
    price: 0.65
  },
  {
    _id: ObjectId("6365147cd18bf6c01b8eeaea"),
    type: 'Golden Delicious',
    price: 0.79
  },
  {
    _id: ObjectId("6365147cd18bf6c01b8eeaeb"),
    type: 'Pink Lady',
    price: 0.90
  }
]

If we wanted to do more advanced queries, such as finding all apples whose type starts with a 'G' and whose price is less than 0.70, we would have to use a combination of query operators. There are many query operators in MongoDB, but some of the most common are:

Going back to the example from before, if we wanted to select all apples whose type starts with a 'G' and whose price is less than 0.70, we could do this:

academy> db.apples.find({
    $and: [
        {
            type: {
                $regex: /^G/
            }
        },
        {
            price: {
                $lt: 0.70
            }
        }
    ]
});
[
  {
    _id: ObjectId("63651456d18bf6c01b8eeae9"),
    type: 'Granny Smith',
    price: 0.65
  }
]

Alternatively, we could use the $where operator to get the same result:

academy> db.apples.find({$where: `this.type.startsWith('G') && this.price < 0.70`});
[
  {
    _id: ObjectId("63651456d18bf6c01b8eeae9"),
    type: 'Granny Smith',
    price: 0.65
  }
]

If we want to sort data from find queries, we can do so by appending the sort function. For example, if we want to select the top two apples sorted by price in descending order we can do so like this:

academy> db.apples.find({}).sort({price: -1}).limit(2)
[
  {
    _id: ObjectId("6365147cd18bf6c01b8eeaeb"),
    type: 'Pink Lady',
    price: 0.9
  },
  {
    _id: ObjectId("6365147cd18bf6c01b8eeaea"),
    type: 'Golden Delicious',
    price: 0.79
  }
]

If we wanted to reverse the sort order, we would use 1 (Ascending) instead of -1 (Descending). Note the .limit(2) at the end, which allows us to set a limit on the number of results to be returned.

Updating Documents

Update operations take a filter and an update operation. The filter selects the documents we will update, and the update operation is carried out on those documents. Similar to the query operators, there are update operators in MongoDB. The most commonly used update operator is $set, which updates the specified field's value.

Imagine that the price for Granny Smith apples has risen from 0.65 to 1.99 due to inflation. To update the document, we would do this:

academy> db.apples.updateOne({type: "Granny Smith"}, {$set: {price: 1.99}})
{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

If we want to increase the prices of all apples at the same time, we could use the $inc operator and do this:

academy> db.apples.updateMany({}, {$inc: {quantity: 1, "price": 1}})
{
  acknowledged: true,
  insertedId: null,
  matchedCount: 3,
  modifiedCount: 3,
  upsertedCount: 0
}

The $set operator allows us to update specific fields in an existing document, but if we want to completely replace the document, we can do that with replaceOne like this:

academy> db.apples.replaceOne({type:'Pink Lady'}, {name: 'Pink Lady', price: 0.99, color: 'Pink'})
{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

Removing Documents

Removing a document is very similar to selecting documents. We pass a query, and the matching documents are removed. Let's say we wanted to remove apples whose prices are less than 0.80:

academy> db.apples.remove({price: {$lt: 0.8}})
{ acknowledged: true, deletedCount: 2 }

Conclusion

By now, you should have a basic understanding of NoSQL databases and how to use MongoDB. The following section will cover some fundamentals of NoSQL injection attacks.