Mulitate4's Programming Madness!

Building True-Do API!

Building True-Do API!

Aryan Dongre's photo
Aryan Dongre
·

8 min read

Welcome to my adventure of building true-do api! The technology used for making the api is - Python, FastAPI and MongoDB.

Why I built this API.

To learn the basics

I had already built a small API for my lyricist website using the http server module in node. Read the article here! Anyways, since I had built an API in js, I wanted to build one in Python too :D. It would also give me some experience on building APIs from scratch!

For a Bigger Project

Also, this API forms a part of a bigger project involving a cross platform mobile and web app with a clean and simple UI for everyday to-do tasks! (That was a mouthful. Oof)

Part 0: Setting up stuff

Choosing the Library

So I researched the popular API libraries and settled on FastApi rather than Flask. (I had tried django earlier and it was quite overwhelming to me :\ . So I wanted to learn the basics first!).

Why I chose FastAPI

The reason I chose FastAPI was that it had much better query string handling, and it was (as the name suggests) FAST. It uses Pydantic to automatically check for data types received by the request, and all-in-all has a better development experience. And as an added bonus, it is used by Microsoft in their Internal API. Pretty cool eh?

Setting Up

Now, I was ready to start building my API! Won't be that hard would it? (Spoiler alert, it kinda sucked, but for reasons different than what you might think). I pip installed the library fast-api and was set! So I started off and realized that I didn't know ANYTHING about fast-api. So, my first and second day was spent looking at examples of simple apps to understand the syntax of fast-api.

Part 1: Starting Out

substantial INSTANTIATION

Starting out was pretty simple. Instantiate your app like this -

from fastapi import FastAPI
app = FastAPI()

Then, to handle a get request for Root (just the URL, in simpler terms)

@app.get("/")
def handler_function():
    return {"hello":"world"}

And voilá, you have your API ready to go!

Understanding this part was easy enough. For those confused, the FUNCTIONS under the decorators (@app.get()) are executed when a GET request is made to that endpoint.

DECORATORS for Decoration

Now the first thing that caught my eye was the use of decorators.@app.get("/") was the decorator for handling a "GET" request from a client. When you enter the URL of an API endpoint. For example: "coolapi.com/bruh", it executes a GET request with the endpoint bruh.

I had already use decorators in building my discord bot - Muli-Bot, so I was already comfortable with using them. (This is also the reason why it was intriguing to me)

A side note - I'm still not a 100% sure how decorators work, but this article helped me clear up some of my misconceptions.

From what I understand, decorators, are used to perform actions on functions, before the functions are executed

Flask vs FastAPI - Decorators.

The Major difference is that in flask, you use

TO BE COMPLETED

Part 2: Getting Into it

DOCUMENTATION radiation

A very swag feature that FastAPI has, is automatic docs! No need to worry about making documentation now! Just build your app (don't forget to add tags) and the library will do the rest for you! To access your docs, add /docs to your root url (example - 127.0.0.1/docs)

MODELS are cool.

FastAPI, provides you with some built-in libraries that actually speed up your workflow. One such library is Pydantic.

Pydantic provides you with models that (in the case of FastAPI) are used to check the data types that are received from your request (data types include - int, str, float, blah, blah)

I had never heard of Pydantic before I started using FastAPI, and at the time, I had no interest in understanding a new library either. So I was resilient at first, and I started looking for an alternative to Pydantic. After a few hours of searching around though, I just decided to check out how Pydantic works, and surprise, surprise! It wasn't that difficult to understand.

A Pydantic Model looks like this - (it's in the form of a class) -

from pydantic import BaseModel

class User(BaseModel):
    username: str
    password: str

And in your API, it would be used this way -

@app.post('/user')
def user_func(user_model: User):
    username = user_model.username
    password = user_model.password
    return {'username':username}

Pydantic returns an error if the data received by the API is not the correct data type. Another cool feature is that it uses built-in data types for type checking!

So once I learnt how to use the Models, I created user models and started work on the API.

Part 3: Data Storage

JSON hey son!

Data storage was kind of a big concern for me. Earlier, I had used .json files to store my data files, and it worked just fine. But for this project, I wanted to do things professionally, so I wanted to use a database.

Now, the problem was that I had absolutely NO experience with working with databases. To tell you the truth, I was scared of databases. So naturally, I just settled with using a .json file for now, then switching to database later.

I already had experience with .json files, so the first stage of development was pretty much a breeze. The flow was pretty much -

Open file -> Authenticate Data -> Make Changes -> Save File -> Return Data

My average function looked like this - 

@app.post("/todo/list")
def add_user(user_model: User):
    todo = open("todo.json", "rw")
    username = user_model.username
    password = user_model.password

    #some authentication logic here. (Checking username and password)

    items = todo[user][todo_items]

And this worked out pretty well. The api was workable. The main problem lied in the switching over to a proper database 😭. I was still determined to understand databases, and to make this api work!

Part 3.5 Data Storage - Database concerns

MONGODB is bongos

A little background: The reason why I (used to) hate databases with a passion was that when I was younger, we were taught MS Access in the most horrific way, leaving me programmatically scarred until I decided to overcome that fear.

I started by looking at the different kinds of databases and the difference between SQL and NoSQL. I had heard of PostgresQL being used in production, usually paired with Django, but I settled on MongoDB because it was purely NoSQL and the idea of freedom appealed to me :V.

I had been watching a course on using MongoDB while having lunch with my brother, so it was a good head-start into the whole DB business. I researched how Python could be connected to MongoDB and found out about Motor (async database connector library ~fancy term for converting python code to MongoDB instructions). I found this cool article I decided to follow.

Everything worked fine. I made a seperate db.py file, and started writing code to connect with my database. Things were going smoothly, perhaps, WAY too smoothly. And of course, the code just decided to NOT work ;-; I actually spent a considerable amount of days looking for a fix to this issue.

LOOP bloopers.

The problem was that MongoDB was trying to use the main loop on which FastAPI was running, and that caused a lot of issues. I started looking for solutions but another problem arose. FastAPI was a fairly recent library and there wasn't a huge community around it, like Flask or Django. So it was VERY hard to find a solution.

What ended up working for me was using the "@app.on_event("startup")" decorator to start the db. The function under it was called when app started. This essentially solved my main problem. So my code looked like this - 

app.on_event("startup")
async def start_db():
    db.start_db()

@app.on_event("shutdown")
async def stop_db():
    await db.close_db()

My database was finally connected, I was so happy! But then I ran into another major problem.

SCOPING out the problem

Now, the problem was that, when I instantiated the DB, it was using an async method and all my CRUD (create, read, update, delete) methods were written in my db.py file, and this file was compiled when it was imported into my main.py file (from where I called the method to start my database connection) and this caused some issues.

The methods didn't pick up the right db instance (due to the problem of variable scopes) and I was stumped. I started looking for a solution, which took a loooong time. Finally there was a completely obscure stackoverflow question, that said that you could attach your motor (mongoDB connector) instance with app.client and it still did not work. But it was definitely promising.

ROUTING away

The next approach I used was that I completely deleted db.py and did the crud in router files themselves. Let me explain - Routers are just a way to separate the code of concern. Like - todo.py and users.py. These are then hooked back to the app in main.py. Here's an example -

#todo.py
from fastapi import APIRouter, Body

router = APIRouter()

@router.post("/list")
async def todo_list(todo: ToDo):
    pass

#main.py

from routes.todo import router as TodoRouter

app.include_router(TodoRouter, tags=["To Do"], prefix="/todo")

Note: todo.py is placed in a folder called routes. Also make sure not to add "/todo/add" as the prefix="/todo" argument already does that. I ran into this issue and felt stupid, so I thought I'd put it here. So routers help you seperate your endpoints according to what they do.

Now, in main, I just added this - 

@app.on_event("startup")
async def start_db():
    app.client = motor.motor_asyncio.AsyncIOMotorClient(MONGO_DETAILS)
    TodoRouter.client = app.client
    UserRouter.client = app.client

now app.client was a motor-Async-IO instance, and that was passed onto TodoRouter. Simple fix and I WAS ECSTATIC that it WORKED! In todo.py - 

def get_collection():
    return router.client.truedo.true_do_collection

Now I could do my cool database login B). The logic took around 2-3 days, since I was new to MongoDB and DBs in general, but I managed to complete everything and was proud of myself!

Finishing Up

There's actually quite some things I need to work on, like adding an update endpoint for updating to-do items, but I will get to them eventually. I'm going to work on a site for this api soon too, and if things go well, learn flutter and build an app! :D.

Anyways, Thanks for reading this far, if you learned anything out of this article, leave a comment, and I'll see you in the next one :)

PythonMongoDBAPIs