Choosing a Database for Your Project: Step by Step

The decision around which database (s) to use is best made at the very initial stage of any project where efficient data management is critical. Experience has shown us that a poor choice will impact your project negatively to the end.

When thinking about which database management system to use, you have to think beyond the data itself. The initial choice determines the cost, security, reliability, and scalability of the entire project. Remember, data collection and storage takes time and can be very costly for some projects. This is why you must always make sure that every decision counts.

At the core of this decision is robust scalability. As a quick demonstration of the importance of scalability, think of a situation where you are looking for a database for a new application. The application is yet to be launched, and as a result, it has limited users. You need to think about what happens when the application grows and attracts millions of users, for example.

The ability to expand and grow with a database should be a forefront consideration. Even so, growth and scalability are not the only factors to consider.

Read on through the steps to get the full scope of what it takes to choose a good database for your project.

Also Read: Relational Databases VS Graph Databases

Step 1: Examine your project requirements

Most developers make the mistake of choosing a database simply because it is popular or well-known.

In reality, the database that works for your project may not necessarily be among the popular solutions in the market. In reviewing your project, there are a few questions you should answer. For instance:

• Will your workload be OLAP or OLTP? We’ll explain shortly in the next step why this is a fundamental consideration.
• What type of data are you dealing with? It could be structured, unstructured, time series, relational, etc.
• Do you intend to scale the project in the future? If yes, consider to what extent the project would need to be scaled.
• What are the write/read patterns for your data? For example, your data may be read-only with occasional writing needs.
• What are the security needs for your datasets? Extra sensitive data, such as credit card transactions, may require extra security measures.
• What is the budget?

Step 2: Make the choice between (OLAP) and (OLTP)

You may have come across this common saying in data science that there are no good or bad databases. This simply means that it’s the unique aspects of your project that will eventually determine what database can be defined as the best. This is where OLAP and OLTP considerations come in handy.

The Online Analytical Processing (OLAP) type of database works better with data that is aggregated. On the other hand, Online Transaction Processing (OLTP) is ideal for transnational data. The aggregation and storage of data from diverse sources, such as websites, applications, smart meters, and internal systems, are facilitated by these systems.

OLAP primarily arranges and groups data to enable analysis from various perspectives, whereas OLTP ensures reliable and efficient storage and updating of transactional data, particularly in high volumes.

Also Read: Differences Between OLAP and OLTP

Step 3: Understand the types of databases available in the market today

Different types of databases offer developers a choice of features that suit their projects. These are the main types to consider.

• Relational Databases: As the name suggests, relational databases are the best for storing related data or one that may require comparison. Such data is commonly structured and may be tabulated. If your project requires a relational data model, you must develop a predefined schema before starting it. In other words, you have to define relationships within the tables. A good example of a relational database is MySQL.
• NoSQL Databases: NoSQL databases are great for big data projects. Are you getting into a project requiring huge data storage and analysis? Then, you may want to consider NoSQL databases. They are different from relational databases in that they are not strictly tabular. They can be document-oriented, graphical, or wide columns. They also do not require a schema, which means they are dynamic and can grow over time. NebulaGraph, the popular open source graph database, is a good example of a NoSQL database.
• NewSQL databases: Considered a revamped version of SQL databases, the idea behind NewSQL databases is to offer big data enterprises room for scaling online transaction processing (OLTP) workloads. This database type is ideal if your project is very big and does require some elements of OLTP.
• Time-series Databases: Let us assume your project’s data is the type that is going to be changing dynamically over time. This is common when building applications aimed for use cases such as long-term climate monitoring, financial markets, etc. In such cases, you want a database that is optimized for tracking time series data. This data model is ideal for projects in fields like IoT.

As the application developer, you should first review your project data and decide the data model that will be most suitable. However, if you feel your project will best suit multiple models, do not hesitate to go this route. You can always consult colleagues and experts to help you decide on the best combination.

Also Read: Differences Between Relational and Non-Relational Databases

Step 4: Evaluate the Write and Read performance needs

After sorting out your options based on the above steps, it is time to consider the speed in terms of write and read performance. Ideally, you will still have to refer to step #2 above. Whether your data suits an OLAP or OLTP system will determine the database to select here.

If you are dealing with data that requires high write speed (can handle constant input and interactions over a unit of time), then an OLAP system is the most ideal. If you are going for a database with high read (high data retrieval within a unit of time), then an OLTP system is ideal.

There are business models that automatically call for high-write databases. For instance, emergency call center systems are always busy with input activities. In such a case, a database with a high write performance is generally advised. The NewSQL data models are better positioned to handle high write and read needs.

It’s also important to carry out performance tests at his level. This way, you will determine the expected latency per unit time, the write and read speeds, and connection capacity, among other performance-related factors.

Step 5: Review security and compliance requirements

A database security breach can be costly for organizations. This is why developers must pay key attention to data security.

In this step, you want to consider factors such as location. For instance, we know there are strict personal data regulations in Europe. If you operate in this region, you may want to be more careful with personal data.

Ideally, most databases are secure and good to use for non-sensitive data. However, in case you are dealing with extra sensitive data such as credit cards, finances, and health, you may have to put in place extra layers of security. Thankfully, some databases guarantee these layers. In this regard, go for a database with features such as encryption, auditing features, and strict access control.

Step 6: Weigh the costs against your budget

The cost will vary depending on the licensing and nature of the database used. For example, open-source databases will not cost much to run and maintain. If you choose a pure commercial option, you must ensure that your budget will be capable of supporting it.

However, the cost and licensing should never be a rigid determinant when selecting a database for a huge and consequential project. You do not want to risk the project just to save a few bucks.

Step 7: Review the need for migration

IBM recognizes source system limitations as one of the major bottlenecks in database migration. In reference to the read-and-write performance above, the quality of database migration will depend on core aspects like the database speed. If you intend to migrate in the near future, consider the read-and-write performance.

Ensure you are going for a database that offers sufficient memory, network bandwidth, and capability for concurrent connections.

Remember that most migrations occur while the database is still active. If you have to shut down operations for a while, you might incur serious losses. This is why you must choose a database that will allow you to migrate data when necessary without affecting operations.

Step 8: Consider ease of use and integration with other technologies

In reviewing the ease of use of a database, there are likely to be obvious biases. This is why it should be purely a personal choice. In most cases, it depends on your background as a programmer or a business person.

There are those who prefer databases that are quite straightforward. For instance, most developers already know what to expect with relational databases. If you have a predetermined pattern of working with data, you may prefer such an option.

On the other hand, there are developers who love evolving with technology. Today, database management systems come with interesting features. If you are among the dynamic individuals who love trying new features, you may want to choose a more dynamic option, such as a graph database or NewSQL databases.

The primary aspects that determine the usability of a database may include the database schema, design, query language, security, and much more.

Another factor that goes hand in hand with usability is integration. You must consider the various applications already being used in the organization that will always need to work smoothly with the application you are building. The database you choose to work with should thus integrate with such apps with negligible effort. For example, there are many CRM tools that are necessary to streamline business operations. A good database should be capable of sharing information and synchronizing with such tools. To this end, databases that support cloud-based operations are highly recommended. If you will need to collect data from different data points such as social media, web applications, and internal departments, go for a data management solution that offers support for this use.

Also Read: Choosing a Database for a Social Media Project

Step 9: Analyze backup and recovery needs

There is no database project that comes without a need for backup and recovery. In the same spirit, all database systems offer some form of backup and recovery solutions. Even so, not all the available solutions are ideal for all projects. Based on your project specifics, you need to consider integral pain points that affect backup and recovery.

For instance, while SQL databases have become a standard for most data administrators, they are not versatile in terms of recovery. One common issue they experience is slow backup and recovery in case of a problem. Choosing a database that supports cloud backup might reduce such constraints.

Step 10: Consider the vibrancy of the community

Before you settle for a database, ask around and find out how many developers use a similar model or even support it. Choose one that is well-known and used by many developers working on projects that are closely related to yours. In such a case, you can easily get assistance from the community whenever you are stuck.

Luckily, all the good databases that we know have online communities. However, how vibrant and active a community is determines how easily you can get help in times of need. Snoop around within forums with mock situations. This way, you will note how quickly you can get help in case you are stuck with a specific challenge. Eventually, settle for a database that is well-backed by people that are always eager to help in addition to understanding the database inside out.

Conclusion

We understand how challenging it often gets when deciding what type of database is suited for a specific project; and we hope that this guide makes things a little easier for you going forward.

Start a free trial of NebulaGraph via AWS or Azure. Take advantage of 3 days free trial on these two clouds and enjoy one-click deployment in the cloud.

Choosing a Database: FAQs

1. When should you go for SQL database solutions?

SQL databases are best suited for projects that involve heavy-duty extraction of data. These databases are quite organized in terms of schema. You should also consider SQL if you are good at the SQL query language.

2. When should you go for NoSQL database solutions?

NoSQL databases are the best for projects that involve heavy-duty interactions. They work better with applications that experience heavy read and write activities such as online retail stores.

3. Can I use more than one database type in one project?

Technically, you can use as many databases in one application as possible. However, factors such as maintenance, data retrieval, backup, and recovery for each database must be considered. Multiple databases are often used in complex projects where multidimensional benefits are sought.

4. What is the difference between the OLAP and OLTP systems?

The primary objective of OLAP lies in analyzing aggregated data, facilitating the generation of reports, conducting intricate data analyses, and identifying trends. Think of what Google Analytics does for businesses. The data has to be collected over time, stored, aggregated and analyzed. Conversely, OLTP focuses on processing database transactions, including tasks like order processing, inventory updates, and customer account management.