Microservices and Message Queues, Part 7: Caveats for real-world use cases

While implementing our demo project in part 5 and part 6, we simplified a couple of things for brevity’s sake. As a result, our demo right now is not a particularly useful product. This article will suggest some things we can do to make our demo more useful and, by extension, fit for real-world use.

Some of the things we need to do to make our demo more usable and useful are, but not limited to:

Adding an interface

Currently, our first service only exposes a REST API endpoint through which users can interact. This will inadvertently preclude non-technical users from accessing our product.

In fact, even for technical users, our demo would only be great for integration with other systems, but not the type of software they could use daily on their mobile devices, for example.

To extend the demo we’ve built so far and make it more usable, we recommend building a web user interface. That way, users can interact with the system via their browsers. There are two ways of approaching this.

First, you can bundle the user interface into the first service - FastAPI supports writing and rendering HTML templates. Alternatively, you can build the user interface as a standalone project with frameworks like React, Vue.js etc.

Making the job search more comprehensive

Remember we are trying to build a system that makes it easier for job hunters to discover relevant jobs. Right now, it does this by connecting to an API that grabs relevant jobs from LinkedIn. Considering that there are tons of job portals out there, it is safe to say that the results we get are not quite encompassing.

We could improve our solution by extending our search beyond just LinkedIn. One way to do this is to connect to more APIs that scrape different job portals. But the problem with this approach is the fact that there aren’t lots of APIs like that.

What we recommend is for you to implement your own custom logic that scrapes about 3 selected job portals. Not familiar with web scraping? You can visit this practical introduction to web scraping in Python to learn more.

Making the job search query customizable

Remember that users submit a search term, a location, and an email to our application, and the application in turn emails relevant jobs matching the search criteria - in this case the search term and location.

However, it is common for job hunters to filter jobs by date - because frankly, it doesn’t make so much sense applying to a job that has been posted three months ago, for example. To make our demo more useful, you can also allow users to add date filters to the search query.

For example, users could specify that they only want to be emailed jobs posted no longer than seven days. Furthermore, you can even make things more convenient for the end user by giving them the option of automating the search process.

In other words, you can implement a feature that uses a user’s search parameters to email them relevant jobs once everyday or twice a week, depending on the user’s preference. That way, users don’t have to repeat the same action - they can just specify the action and specify how often they want it executed - more like cron jobs.

One way to implement that cron functionality is to to store a user's first API request in a DB, at the same time as it's added to the queue. We could then run cron jobs on that db, and do a new search every day(if the users wants it done everyday) and email the result to the user.

Deploying the project

Lastly, the demo project you implemented currently only lives on your local machine. As a result, only you can interact with the application. For other users to be able to use our demo, we’d have to deploy it to the interwebs.

Luckily for you, I’ve written a detailed guide that demonstrates how to set up a CI/CD pipeline that could automatically deploy your FastAPI project to AWS using Semaphore. Alternatively, you can also checkout my guide on deploying to AWS with GitHub Actions. Even though the demo project in that tutorial was written in Django, you can easily make it work with your FastAPI project as well.

Wrap up

If you’ve made this far then we are super proud of you 🙂- It’s been such a long series. To help us digest the concept of message queues the first half of this series focused on demonstrating how message queues foster asynchronous communication between two or more processes.

Furthermore, we also went ahead to show how this inherent ability of message queues to foster asynchronous communication makes them great for use in distributed architectures. Lastly, we ended the first half of this series by looking at RabbitMQ, and how CloudAMQP eliminates the task of managing your own RabbitMQ server - serverless is a thing now, anyway xD.

To advance your understanding of message queues and, more specifically, RabbitMQ beyond just information, we worked on a demo project in the second half of this series. The demo project allowed us to apply some of the concepts we had learned throughout the series to an actual project.

This is the first iteration of this series and we intend to continuously make it better. We’d be happy to hear from you! Please leave your suggestions, questions, or feedback in the comment section below or get in touch with us at contact@cloudamqp.com.

CloudAMQP - industry leading RabbitMQ as a service

Start your managed cluster today. CloudAMQP is 100% free to try.

13,000+ users including these smart companies