Modular monolith blueprint

Harnessing simplicity and efficiency: A deep dive into our FastAPI Modular Monolith BluePrint

Apr 19, 2024

For years, we were all in a bubble trying to solve most of our use of micro user problems.

Nowadays, the drawbacks of microservices, such as increased complexity, make other approaches more suitable.

The modular monolith approach is gaining some serious momentum because it hits the sweet spot between the traditional monolithic architecture's simplicity and microservices architecture's flexibility. It offers a scalable, straightforward way to develop applications, making it particularly appealing when rapid development and deployment are essential.

Our Journey with FastAPI

Our FastAPI Modular Monolith Blueprint is a testament to the framework's versatility and ability to support scalable, efficient, and, importantly, maintainable web applications. This blueprint is designed to give you a solid foundation, propelling your projects into the future.

This article will cover up-to-date FastAPI modular monolith architecture:

explore the FastAPI key features and how it stands tall among other frameworks
break down the modular monolith architecture
understand why it’s a robust strategy for modern application development
start with the ideal project structure
cover the advanced features and practices: testing and scalability deployment
explore the tools you need to build with confidence and clarity.

👉 If this sounds valuable, subscribe to receive the following posts in your inbox

Understanding FastAPI

FastAPI at a Glance

FastAPI is a modern web framework focused on high performance. It is designed specifically for building APIs, leveraging standard Python type hints.

Why FastAPI?

So, what makes FastAPI the go-to choice for a modular monolith approach? Let's break it down:

Performance: It's fast. Fast. Thanks to Starlette for the web parts and Pydantic for the data parts, it's one of the fastest web frameworks available.
Ease of Use: Its design, heavily influenced by Flask, makes it intuitive for developers while providing the advanced features you'd expect from a modern framework.
Automatic API Documentation: FastAPI generates interactive API documentation automatically. This means less time writing documentation and more time coding.
Type Hints: By leveraging Python's type hints, FastAPI ensures cleaner code that is less prone to errors.

Compared to the Rest

When placed side by side with other popular frameworks like Flask and Django, FastAPI shines for API-centric applications.

While Flask offers simplicity and flexibility and Django provides a robust full-stack batteries-included framework, FastAPI stands out for its asynchronous capabilities and automatic API documentation.

FastAPI offers a sweet spot for developers looking for speed, efficiency, and scalability, all wrapped up in a user-friendly package. It's a testament to how far web development has come and a nod to where it's heading.

Please read along to see how FastAPI elevates this architecture, making it future-proof.

Modular Monolith Architecture

Let's unpack the concept of modular monolith architecture, which feels more like assembling a LEGO set than tackling a daunting tech puzzle.

The Best of Both Worlds

The charm of the Modular Monolith approach is like enjoying the simplicity of a single-story house while revealing the distinctiveness of each room.

This approach is gaining traction for several reasons: scalability, simplicity, and the ability to rapidly develop and deploy services without getting bogged down in the complexities often associated with microservices architectures.

Harness the simplicity and ease of a monolithic structure without sacrificing the separation of concerns offered by modules. Think of it as managing a collection of interrelated yet independently functioning mini-applications within a larger ecosystem.

Why Modular with FastAPI?

Here's why using FastAPI with a modular monolith can elevate it:

Structured Yet Segregated: FastAPI's ingenious use of routers and dependency injection mechanisms is the perfect conduit for creating distinct yet interconnected modules. Each component remains an island of functionality, making the entire application more accessible to navigate, debug, and maintain.
Scalability on Your Terms: Start simple and expand when needed, module by module, without constant refactoring. You can scale your application horizontally as your user base or feature set expands.
Maintain Speed, Gain Efficiency: Leverage FastAPI's perks of high-speed response times and optimal performance to ensure your application scales smoothly.

Crafting Your Modular Monolith with FastAPI

Venturing into constructing a modular monolith with FastAPI might seem daunting but surprisingly straightforward. Here's a scaffold to get you started:

Blueprint Your Application: Identify the core functionalities (modules) your application will need. This could range from user authentication to payment processing, each serving as a standalone module within the larger framework of your application.
Use Routers and Dependency Injection: FastAPI's routers allow you to neatly organize your API endpoints by module, ensuring a clean separation of concerns. Dependency injection, on the other hand, helps you weave standard functionalities across these modules, eliminating redundancy and fostering code reuse.
Fortify Your Core: The shared kernel, or the heart of your modular monolith, should be robust. It should house standard utilities and functions that are essential across different modules. It's the glue that holds your application together, ensuring consistency and efficiency in operation.

The Strategic Edge

Opting for a modular monolith architecture is a strategic decision. It positions you to evolve and adapt your application with agility, leveraging FastAPI's strengths to streamline development and enhance performance.

Crafting a Modular Monolith with FastAPI

Imagine we're building Conduit, a clone of Medium, but with our twist. Like any robust application, Conduit needs a solid foundation, and that's where FastAPI's Modular Monolith Blueprint comes into play.

We're not just going to talk about theories; we'll apply them to real-world scenarios. Let's see how this blueprint transforms Conduit from a mere idea into a structured, scalable application.

Structuring Conduit with FastAPI

At its heart, Conduit is about sharing ideas through articles, interacting with them through comments, and connecting with others. To manage this effectively, we break down Conduit into several modules: Users, Articles, and Comments. Each module represents a core functionality of our application.

Project Structure

In FastAPI, our project structure is the backbone of the modular monolith. For Conduit:

Users Module: Handles user registration, authentication, and profiles.
Articles Module: Manages article creation, updates, listing, and categorization.
Comments Module: Deals with adding, deleting, and viewing article comments.

Each module acts like a self-contained mini-application, complete with its models, schemas, services, and routes. This separation ensures that changes in one module don't ripple uncontrollably through others.

Shared Kernel and Service Layer

Shared Kernel: Consider it the communal area where all modules can pick up the necessary tools and utilities. It includes user authentication utilities, base model definitions, and standard exceptions. It's a central library where every module checks out what it needs but returns it for others to use.

Service Layer: Each module has its service layer, a middleman that interprets the complex language of business logic into simple operations.

For instance, the Articles service layer might handle operations like "publish article" or "archive article," which involve more than just a straightforward database call.

Implementing Conduit with FastAPI

Here’s a quick peek into how:

API Routers: Each module has its API router. The Articles module, for example, would have routes for posting a new article, fetching articles, etc. FastAPI's routers make keeping these endpoints neatly organized within each module easy.
Dependency Injection: FastAPI’s dependency injection system is a game-changer for sharing standard functionality. The database connection, for example, is injected into routes that need it, ensuring we don't repeat ourselves or bloat our code.

The Real-World Benefits of FastAPI

By adopting a modular monolith architecture with FastAPI, Conduit benefits from:

Scalability: As Conduit grows, adding new features or scaling existing ones becomes a matter of enhancing or adding modules. The rest of the application remains undisturbed, maintaining stability.
Maintainability: With clear boundaries and responsibilities, maintaining Conduit, updating features, or fixing bugs becomes significantly easier. Developers can focus on specific modules without needing a map of the entire application.
Performance: Thanks to its asynchronous support and speed, FastAPI ensures that Conduit scales are maintained well and perform exceptionally.

This approach offers a clear pathway to evolving Conduit over time, adding new features, or transitioning to microservices if needed. All while keeping our codebase clean and our development process efficient.

Advanced Features and Practices

Database Enhancements and Multi-Tenancy

Imagine Conduit is now bustling with users, articles, and comments from different organizations or user groups. Each group expects their data to be isolated and secure from others. This is where multi-tenancy comes into play.

FastAPI doesn't dictate how you handle your database, allowing you to implement multi-tenancy in a way that best suits your application, whether through separate databases, schemas, or a shared database with tenant ID columns.

Please ensure your data layer can handle these separations without complicating your business logic.

Asynchronous Communication with Kafka

As Conduit grows, efficient communication between its components becomes crucial. Enter Apache Kafka, a powerhouse for handling real-time data streams.

Kafka allows different parts of Conduit, such as the article publishing system and the comment notification system, to communicate asynchronously and reliably. This decouples our modules, allowing them to scale independently and handle loads more efficiently.

Integrating Kafka with FastAPI, we can ensure that a surge in comments on a viral article only bogs down part of the application, maintaining smooth operation across the board.

Implementing Advanced Features in Conduit

Implementing these advanced features in Conduit with FastAPI leverages technology smartly while controlling complexity.

Here’s how these features translate into tangible benefits for Conduit:

Robust Data Handling: By adopting advanced database patterns and multi-tenancy, Conduit can offer tailored experiences to different user groups while keeping their data secure and isolated. This enhances user trust and allows for the customization of content and features per tenant.
Efficient Module Communication: Kafka facilitates the asynchronous data exchange between Conduit's modules so that the application can handle high volumes of interactions without a hitch. This means that our users get a responsive, reliable platform where their actions, whether posting an article or commenting, feel instantaneous and seamless.

Embracing these advanced features and practices in FastAPI is like equipping Conduit with superpowers. However, with great power comes great responsibility.

When implementing these advanced features, please remember to balance innovation with the practicality of your application's needs and growth stage.

Always weigh the benefits against the complexity these features might introduce.

The goal is to build a Conduit that is scalable, maintainable, and, above all, valuable to its users.

Stay grounded in the principles that make our software robust and our user experiences delightful.

Testing Strategies

Testing gives developers confidence, and it's the backbone that ensures our Conduit app stands tall and robust for all user interactions and updates it will face.

FastAPI's TestClient simplifies testing our API endpoints to the point where it almost feels like cheating.

The Magic of FastAPI's TestClient

Imagine you've just brewed your morning coffee and settled in, and you're ready to ensure that the new features in Conduit are sturdy.

TestClient is built into FastAPI, and it lets you call your API routes just as you would in the real world without the messy deployment steps. It's like having a rehearsal in the safety of your development environment.

Why It's a Game-Changer:

Simplicity: Setting up TestClient is straightforward. You define your FastAPI app instance, pass it to TestClient, and you're ready to roll.
Flexibility: Whether you're testing a GET request to fetch articles or a POST request to create a new comment, TestClient handles it all, mimicking real-world requests without the overhead.
Accuracy: What you test is what you get. TestClient's responses mirror what your users will experience, allowing you to catch and squash bugs early in the development cycle.

Testing Conduit API

Testing Conduit ensures the code runs and crafts experiences that users love and trust. Here's how we can apply TestClient in our workflow:

Endpoint Testing: For every Conduit route– signing up a new user or posting an article – we write tests to simulate requests and inspect the responses for correctness.
Authentication and Authorization: With features like JWT authentication in place, we use TestClient to ensure that our security measures are not just present but impenetrable.
Performance Under Load: While TestClient focuses on functionality, we also integrate performance testing tools to simulate high traffic, ensuring Conduit remains swift and responsive.

Best Practices for Comprehensive Test Coverage

To get the most out of our testing efforts, we follow a set of best practices that ensure every nook and cranny of Conduit is examined:

Automate Everything: Each time a new piece of code is added or an existing feature is modified, automated tests run, keeping our codebase healthy and our developers happy.
Think Like a User: We craft tests to mimic actual user behaviors, covering typical and edge-case scenarios, ensuring Conduit remains user-centric.
Continuous Integration (CI): By integrating our tests into a CI pipeline, we catch issues early before they make their way into production.

Having a robust testing strategy is non-negotiable. It encourages a culture of quality and reliability, from the smallest code unit to the full spectrum of user experiences.

Scalability and Deployment

Two topics that can either make or break our beloved Conduit app. Now, we're in the thick of it, the real deal. How do we ensure that as our user base grows, Conduit doesn't just buckle under pressure but thrives?

Scaling Conduit: A Balancing Act

Imagine you're at a buffet. Initially, it's just you, so the food is plenty, and you're back at your table in no time. Now, imagine a busload of hungry tourists arriving. Suddenly, the queue is long, and the food replenishment can't keep up. That's your app without scalability.

FastAPI gives us a solid foundation with its async capabilities and speed. But as our user base grows, so do the requests. Here's our game plan:

Divide and Conquer: We modularized Conduit for a reason. Each module can scale independently based on demand. Think of it as having separate queues for each dish at the buffet.
Embrace the Cloud: Cloud services like AWS, Google Cloud, and Azure offer auto-scaling capabilities. They're like buffet managers who bring out more food or open new buffet lines as the crowd grows.
Cache Wisely: Sometimes, users keep coming back for the same dish. Caching popular content reduces the load on our servers, ensuring everyone gets their fill without waiting too long.

Deployment: From Local to Global

Deployment is the moment of truth, where we set Conduit free into the wild. It's like opening night for a play; everything needs to be just right. Here's how we ensure a standing ovation:

Containerization with Docker: Packaging Conduit into Docker containers makes it portable and consistent across any environment.
CI/CD Pipelines: Continuous Integration and Continuous Deployment ensure that every new feature or fix is automatically tested and deployed without manual intervention.
Monitoring and Alerts: Once deployed, we monitor Conduit's performance and health with tools like Prometheus and Grafana.

The Cloud as Our Ally

Cloud platforms are reliable partners in Conduit's growth. They offer managed databases, serverless functions, and more, which can be leveraged to enhance Conduit's scalability and resilience.

Scaling and deploying an app like Conduit isn't a one-time effort; it's an ongoing journey of adaptation and improvement.

We've set the stage for success with FastAPI's performance, Docker's consistency, and the cloud's scalability.

Moving Forward

What does the future hold for our Conduit app, and how will the path toward microservices unravel if and when we decide to take that route? Equally importantly, what are the future trends in FastAPI and modular monoliths to which we should keep our antennas tuned?

The Evolution of Microservices

Your project starts to grow, and suddenly, the monolith that served you so well shows its limits. The beauty of starting with a modular monolith, especially with FastAPI, is that it's like having a set of Legos. Each module, or "Lego block," has the potential to become its microservice when the time is right.

When to Make the Leap: Listen, no alarm goes off telling you it's microservice time. It's more about feeling the strain — maybe deployments are getting cumbersome, or the team is stepping on each other's toes too often. That's your cue to consider separating a module or two into separate services.
Key Considerations: Before breaking modules apart, consider the complexity you're adding. Each microservice will need its own deployment pipeline and monitoring.

Future Trends: Keeping Up with FastAPI and Beyond

FastAPI is evolving and growing more robust and flexible. Here's what's on the horizon:

Performance Enhancements: FastAPI is already fast (duh!), but the community is always finding ways to reduce milliseconds. Look for updates that make it even zippier.
Better Integration with Frontend Frameworks: The line between the backend and frontend is blurring. Expect FastAPI to offer more seamless integrations with frameworks like React and Vue, making full-stack development smoother.
AI and Machine Learning: With Python being the darling of the AI/ML world, watch for FastAPI to become a go-to for serving up machine learning models and handling heavy data processing tasks.

Continuous Learning and Adaptability

The only constant in technology is change. Embrace it.

The modular monolith approach with FastAPI gives you a flexible, robust foundation, but the learning continues.

Conclusion

We've journeyed through the essence of building a scalable, efficient, and maintainable web application using FastAPI's Modular Monolith Blueprint. Conduit is just a starting point; the real adventure begins with your projects and ideas.

Experiment with FastAPI, twist it, turn it, and see how it fits into your vision. The modular monolith approach is a powerful ally in your quest to build something unique.

Resources for the Road: Before you set off, arm yourself with knowledge. The FastAPI documentation is your scripture. Forums like Reddit's r/FastAPI and GitHub discussions are your fellowship. And remember the tutorials scattered across the web like hidden treasures waiting to be discovered.

The future is modular. The future is FastAPI. Let's build it, one module at a time. 🚀

We've covered a lot of ground together, haven't we? It's been quite the journey from the speedy lanes of FastAPI to the structured blocks of the modular monolith architecture. If there's one thing you can take away from this, it's that building scalable, efficient, and maintainable web applications isn't just about choosing the right tools; it's about understanding how to use them effectively.

FastAPI has proven itself to be more than just a fast web framework. It's a gateway to building applications that can grow with your ambitions, whether you're a solo developer or part of a burgeoning tech team. Its design caters to speed and the future of web development, making it a valuable skill in your toolkit.

Reflection on Modular Monoliths: We delved into modular monolith architecture, not as a buzzword, but as a practical approach to developing complex applications with simplicity at their core. The beauty of this architecture lies in its flexibility — it's like having the structure of a monolith with the agility of microservices.

A Call to Action for Innovators: Now it's over to you. The FastAPI Modular Monolith Blueprint is more than a concept; it's a call to action. It's an invitation to challenge the norms, innovate, and build web applications that are fast and functional and a joy to work on and evolve.

Resources for the Road Ahead: The journey doesn't end here. The world of FastAPI and modular monoliths is vibrant and ever-expanding. The FastAPI documentation is your first port of call to look at. Community forums and advanced tutorials are your map and compass, guiding you through challenges and inspiring new ideas.

In wrapping up, I encourage you to experiment, learn, and, most importantly, share. The tech community thrives on collaboration and the exchange of ideas. Whether it's a blog post, a GitHub repo, or a chat over coffee with fellow developers, your insights and experiences contribute to the collective knowledge and innovation in the field.

FastAPI and the modular monolith architecture offer a solid foundation, but the community — it's you — will drive the future of web development. So build something amazing, and let's shape the future together. 🚀

🎯 More on my Substack

If you like this, please check my Substack for practical tips on scaling your startup and additional resources.

At CodeSwiftr, we are exploring creating a complete step-by-step course/tutorial on implementing a Conduit medium clone API using our above blueprint.

Is anyone interested in such a paid course? What would you want it to include?

🎯 Actionable advice and additional Resources

I'm sharing a few tips and resources that have helped me in my career. These are based on my experiences in the tech field. They can help you understand and plan for tech stacks better.

To make the most out of FastAPI and the modular monolith architecture, consider these strategies:

Embrace a Modular Design: Focus on creating separate modules within your application to keep it scalable and maintainable. This approach allows you to manage and update parts of the application independently without affecting the whole system.
Invest in Testing: Integrate comprehensive testing to ensure robustness and reliability. Use FastAPI's testing capabilities to simulate real-world usage and catch potential issues early.
Optimize Data Handling: Use FastAPI’s dependency injection to efficiently manage data flow and service interactions. This helps optimize your application’s performance and scales well with increasing data volumes and user demands.
Prioritize Security: Implement security best practices within each module. FastAPI provides tools to secure your application quickly, including security schemes that can be integrated into your API routers.
Keep Learning and Updating: Stay updated with the latest releases and features in FastAPI and the surrounding ecosystem. I'd like you to regularly refine your skills and update your application to leverage new optimizations and capabilities.

Curated Resources to Advance Your FastAPI Journey

To further your understanding and mastery of FastAPI and modular monolithic architectures, delve into these resources:

“Building Microservices” by Sam Newman: Provides insights into designing and building microservices, which can complement your understanding of modular architectures. Although it focuses on microservices, the principles can enhance your approach to modular monoliths.
FastAPI Documentation: An essential resource that offers comprehensive guides, quickstart tutorials, and detailed API references. Perfect for both beginners and advanced users looking to sharpen their FastAPI skills.
“Designing Data-Intensive Applications” by Martin Kleppmann: Helps you understand the intricacies of building applications that manage significant amounts of data, which is crucial for effectively designing modules that interact and manage data efficiently.
Python Type Hints: Enhance your knowledge about Python’s type system, which FastAPI utilizes to ensure more robust code and better developer experience.

More next week

Subscribe below for a future article where we delve deeper into effective tactics for avoiding over-engineering with a focus on the following:

Over-engineering in Startups: The effect of excessive complexity on innovation and resource utilization.
We are identifying over-engineering, which is how to recognize and diagnose over-engineering in projects.
Strategic Simplification: Key principles for avoiding software architecture and project management complexity.
Agile and Lean Transition: Discover how lean principles improve projects, goals, feedback, and efficiency.
Leadership Against Over-engineering: The influence of leaders in promoting simplicity and clarity.

Join us next week as we continue our exploration into mastering simplicity in technology projects, ensuring that our innovations are not only cutting-edge but also practically tailored to meet real-world needs.

✌️ Thanks!

That's all for today! I would love to hear from my readers (and if you've made it this far, you're one of the bravest). Please don't hesitate to contact me on LinkedIn or Twitter and message me. I always respond to everyone!