0%
March 29, 2022

Mongoose in Nodejs and Mongoengine in Python

mongo

nodejs

python

Contents

Connection

  • In mongoose.

    // getConnection.ts
import mongoose from "mongoose"

let connCache: typeof mongoose;

export default getConnection = () => {
  if (!connCache) {
    const dbOptions = {
        user: config.get('db.user'),
        pass: config.get('db.password'),
        maxPoolSize: config.get('db.poolSize'),
        ssl: config.get('db.ssl'),
        replicaSet: config.get('db.replicaSet'),
        authSource: config.get('db.authSource'),
        useUnifiedTopology: true,
        useNewUrlParser: true,
    };
    connCache = await mongoose.connect(config.get('db.uri'), dbOptions);
  }
  return connCache
}

    Example of corresponding mongo db config:

    {
  "uri": "mongodb+srv://some.net/dbName?retryWrites=true&w=majority"
  "user": "user"
  "password": "password"
  "authSource": "admin"
  "ssl": true
}

    With mongoose being a singleton, after monngoose.connect() we can freely import our Model objects and do our CRUD operations using these objects.

    Use of the Connection Object. Sometimes it is more convenient to get the model by just using the modelName , i.e., conn.model(modelName), and this is the reason we export the method to get conn.

  • In mongoengine. After pip install mongoengine in our virtual environment, we can start connecting to our db as follows:

    from mongoengine import connect
from config2.config import config

# if our os.environ["ENV"] == abc, make sure config/abc.json (or yaml) exists
# with the field "db": { "name": ..., }
db_config = config.get("db");

connect(
    alias=db_config["name"],
    host=db_config["uri"],
    username=db_config["user"],
    password=db_config["pwd"],
    authentication_source=db_config['auth_source']
)

Schema Examples and DataTypes

  • In mongoose. The datatypes used in a schema are simply.

    • String
    • Number
    • Boolean
    • [subSchema]
    • [String]
    • [{id: String, name: String}]

    Recall that String, Number and Boolean are default classes built-in in javascript.

    Note that a sub-schema (i.e., a schema of the sub-document/embedded-document) will have _id generated by default. If we just want an ordinary object, make sure we have _id: false as below:

    const subSchema = new Schema<T>({...}, {_id: false, timestamps: true})

    or otherwise just use an inline-schema as simple as {id: String, name:String}.

    Finally we raise an example of a schema:

    export const OilModelName = "Oil";

const schema =
  new Schema() <
  TOil >
  ({
    id: String,
    name: {
      english: String,
      latin: String,
      chinese: String,
    },
    symptomToTreat: [String],
    type: String,
    isDeleted: Boolean,
  },
  {
    timestamps: true,
  });

export const OilModel = model(OilModelName, schema);

  • In mongoengine. The class attributes inside a subclass of Document actually define the schema for that document.

    Generally a schema in mongoengine looks like:

    from mongoengine import Document
from mongoengine.fields import FloatField, DateTimeField
from datetime import datetime

class VibrationSensorMetrics(Document):
    meta = {
      "collection": "vibrationsensormetrics"
    }
    rmsVelocity = FloatField()
    createdAt = DateTimeField(default=datetime.utcnow())

    Due to the value in meta, a new document will be saved in a collection named automatically as vibrationsensormetrics (lowered cases + "s")

    We have the following fields that define the type of each "key" in a document:

    • StringField
    • DictField
    • BooleanField
    • IntField
    • ListField
    • DateTimeField
    • ObjectIdField
    • FloatField
    • EmbeddedDocument
    • EmbeddedDocumentField
    • DictField Among the fields above, DictField is used only when we know it is an object but don't know the data structure. For known structure, we use EmbeddedDocumentField and we explain it in the next section.

Complicated Example with Nested Objects in mongoengine

We continue to complexify the schema of VibrationSensorMetrics above, we will use all of the datatypes defined in the previous section.

from mongoengine import Document
from mongoengine.fields import (
    StringField, BooleanField,
    IntField, ListField, DateTimeField,
    ObjectIdField, FloatField, EmbeddedDocument,
    EmbeddedDocumentField
)
from datetime import datetime


class FreqVelocity(EmbeddedDocument):
    frequency = FloatField()
    velocity = FloatField()


class SpecificFreqVelocities(EmbeddedDocument):
    ft = EmbeddedDocumentField(FreqVelocity)
    rpm = EmbeddedDocumentField(FreqVelocity)
    bsf = EmbeddedDocumentField(FreqVelocity)
    bpfo = EmbeddedDocumentField(FreqVelocity)
    bpfi = EmbeddedDocumentField(FreqVelocity)


class VibrationSensorMetrics(Document):
    meta = {
        'collection': 'vibrationsensormetrics',
        'indexes': [
            ('sensorId', 'measurableId', 'workspaceId'),
            ('sensorId', 'measurableId')
        ]
    }

    def save(self, *args, **kwargs):
        if not self.createdAt:
            self.createdAt = datetime.utcnow()
        self.updatedAt = datetime.utcnow()
        return super(VibrationSensorMetrics, self).save(*args, **kwargs)

    _id = ObjectIdField()
    sensorId = ObjectIdField()
    measurableId = ObjectIdField()
    measurableComponentId = ObjectIdField()
    workspaceId = ObjectIdField()
    direction = StringField()
    s3ObjectKey = StringField()

    freq = ListField(FloatField())
    acceleration = ListField(FloatField())
    velocity = ListField(FloatField())
    stage = IntField()
    rmsVelocity = FloatField()
    specificFreqVelocity = EmbeddedDocumentField(SpecificFreqVelocities)

    startTime = DateTimeField()
    endTime = DateTimeField()
    createdAt = DateTimeField()
    updatedAt = DateTimeField()

    archived = BooleanField()

The EmbeddedDocumentField is used to embed a document (without ObjectId) into that key.

An example of the above schema:

{
  "_id": {
    "$oid": "6243d2cfb6c5eb79cbded8dc"
  },
  "sensorId": {
    "$oid": "623a8e98607fd5d3f2061412"
  },
  "measurableId": {
    "$oid": "623bcb04dc39ae10e39c7857"
  },
  "measurableComponentId": {
    "$oid": "623bcb04dc39ae10e39c7858"
  },
  "workspaceId": {
    "$oid": "62219400a6533f774a0a8500"
  },
  "direction": "radial",
  "s3ObjectKey": "ICC_Vibration/Erbessd_Phantom/189280107/20220304/0504_Ch1.txt",
  "freq": [],
  "acceleration": [],
  "velocity": [],
  "stage": 0,
  "rmsVelocity": 0.7,
  "specificFreqVelocity": {
    "ft": {
      "frequency": 6.346813346174303,
      "velocity": 0.6951668919677223
    },
    "rpm": {
      "frequency": 14.166666666666666,
      "velocity": 0.3500187046119141
    },
    "bsf": {
      "frequency": 56.86395840490943,
      "velocity": 0.05195638432785299
    },
    "bpfo": {
      "frequency": 139.62989361583465,
      "velocity": 0.01651277478295155
    },
    "bpfi": {
      "frequency": 172.036773050832,
      "velocity": 0.007293205675754226
    }
  },
  "startTime": {
    "$date": "2022-03-03T21:04:00.000Z"
  },
  "endTime": {
    "$date": "2022-03-03T21:04:05.000Z"
  },
  "timestamp": {
    "$date": "2022-03-30T03:47:25.852Z"
  }
}

Operations

Basically in mongoose we rely on the Model instance and in mongoengine we rely on the Document instance.

Init Models/Documents, Create and Save Documents, Collection Names

  • In mongoose. We create/init a Model object by either

    import { Schema, model } from "mongoose";

type TCustom = { id: String, name: String, isDeleted: Boolean };
const schema =
  new Schema() <
  T >
  ({ id: String, name: String, isDeleted: Boolean }, { timestamps: true });

export const SymCatModel = model(SymCatModelName, schema);

    or

    // create connection
const conn = await mongoose.connect(config.get("db.uri"), dbOptions);
// init model
conn.model("TheModel", theModelSchma);
// get the model
model = conn.model("TheModel");

    and use the async methods:

    • Model.create(object: TCustom) or
    • Model.insertMany(objects: TCustom[]) to create documents (don't need to call .save() method).

  • In mongoengine. We create/init a Document instance, the instance is a callable function that can create a document:

    vsm_doc = VibrationSensorMetrics()
new_doc = vsm_doc(
    rmsVelocity = 0.1
)
new_doc.save()

    A new document will be saved in a collection named vibrationsensormetrics.

Query for Documents

  • In mongoose. We use:

    • Model.find(filterQuery)

  • In mongoengine. We can either pass a keyword argument or a raw dictionary as a filterQuery to a Document instance:

    • Document.objects.get(_id=some_id)
    • Document.objects(_id=some_id).order_by("-timestamp").first()
    • Document.objects(__raw__={ "_id": some_id }) For example, we can assign vsm_doc = VibrationSensorMetrics() with vsm_doc in place of Document in the above methods.
Update Documents

  • In mongoose.

    // or updateMany
const result = await OilModel.updateOne(
  { id: oilUpdate.id },
  { $set: oilUpdate },
  { upsert: false }
);

  • In mongoengine.

    vsm_doc = VibrationSensorMetrics()
vsm_doc.objects(_id=some_id).update(__raw__={
    "$set": {
        f'measurableComponent.{cpn_index}.velocity.value': highest_velocity,
        f'measurableComponent.{cpn_index}.velocity.latest': time
    },
    "unset": {
        "measurableComponents": ""
    }
},
    upsert=False
)
Delete Documents

For the sake of completeness we also mention how to delete documents. Although in practice we prefer assigning a boolean to soft-delete a document instead.

  • In mongoose. 
    Model.deleteMany(filterQuery);
  • In mongoengine. 
    Document.objects((__raw__ = filterQuery)).delete();