One of the possible reasons for a decrease in the speed of on-premise Bitrix24 is always the growth of the system database. This is inevitable, because over time the client base grows, documents stored in the system, archives of communication history and much more. The problem becomes especially noticeable after a long time of use. Bitrixoids was contacted by a customer who has been working at Bitrix24 for the past 10 years, during which the performance has dropped to critical levels. But we fixed everything.
Briefly about the situation
During the period of use, more than 7,000,000 records were accumulated in the CRM only for the Entity “Lead”, which is quite a large number by the standards of Bitrix24. The on-premise edition underwent many customizations specifically for the customer’s business processes and was updated to the latest version, but over time, the problem of speed came to the fore: every request from employees to CRM caused an increasing load on the system. For users who had more than 100 in the company, it looked like a long delay between action and response – up to 7 minutes of delay after any operation. We will remind you that usually more than 30 seconds of delay results in disconnection with the server with error code 504. However, it happened more tragically at the software level, when the entire RAM of the server part experienced constant and long-term overloads.
Next, you will learn why neither updating the on-premise edition of Bitrix24 nor increasing server capacity helps in this situation. However, our client made a difficult decision to delete a large part of relatively old contacts in order to reduce the size of the database. Fortunately, after studying the situation, Bitrixoids experts managed to find an alternative solution.
The impact of database growth on the work of Bitrix24
For the best understanding of the essence, we offer a simple explanation of the operation of the Bitrix24 database. Other CRMs work in almost the same way, because in a narrow sense, all these systems are a set of various data related to each other. The Bitrix24 database consists of many tables that are filled unevenly and depending on the direction of the company’s work. However, if you are actively working with a huge client base, it is natural that the most data will go to those tables that contain information about leads, contacts, companies, subsequent deals with them, etc.
However, some tables grow unevenly over time, but the number and relationship between different table fields is important. So when an average user of Bitrix24, for example a company manager, opens the Leads section of CRM, the system generates a series of queries to the database that should find, read and display the information the person needs. Requests are processed by the server’s random access memory (RAM) and processor (CP), the resource of which is always limited. The system reads all data fields in the database tables and includes unevenly extended tables. In our case, some tables with millions of records accumulated by the client amounted to more than 60 GB of permanent memory. So now you can imagine what computational effort the system must allocate to process just 1 request from 1 user for just 1 entity.
Bitrix24 updates will not help
Some believe that the upgrade version of the Bitrix24 editor can affect the performance of the system. In general, yes, because each new version is the correction of previous errors and the introduction of new technologies that improve certain aspects. However, the principle of working with the database remains unchanged, so in our case nothing has fundamentally changed. The logic of working with the Bitrix24 database is practically no different from other products and is already optimal for the world of digital technologies.
Server boosts are not effective
Any server has its own maximum capacity, which even this complex equipment cannot exceed. The issue of memory overload has always faced IT specialists, but today its solution has reached its maximum potential. So a banal increase in RAM, let’s say twice (from 32 to 64) cannot significantly affect the situation. Even a larger amount of memory will instantly be filled with a request to a huge database and the system will “hang” as before. Our specialists calculated that in the client’s case, increasing the RAM would speed up the response by a maximum of 1 second, which is extremely high by technical standards, but practically makes no sense to the user. In addition, the increase in capacity significantly affects the cost of the server. So the illusory success in shortened long-term responses is not comparable to economic indicators.
Solution by Bitrixoids
There is a long-known rule: “If you are not able to change the situation, change your approach to it.” After a detailed study of the client’s business processes, our team came to the conclusion that the old adage really makes sense. So now the developers faced an important and responsible task – how to customize the system in such a way as not to harm the general logic of Bitrix24. In general, this is a challenge facing all developers of ready-made products, which Bitrix24 is. After all, creating any new separate solution is not as difficult as trying to implement in the code the expansion of opportunities for someone else’s development. Thanks to expert knowledge and many years of experience, our team succeeded in this.
First of all, new tables were created for recording lead data with the working name “Reserve”, duplicating the structure of the original tables (in fact, these are several tables, but these are already technical details that are not of particular importance). From the old large database, data on irrelevant leads with which there was no activity in the last few years moved here.
The next step was to intercept requests in the automation processes of the company’s existing work flow to the live table and distribute the load on the data. In this way, the work of the entire Bitrix24 ecosystem is reproduced in the usual mode, but without recourse to a large volume of backup data. It is worth noting that the new approach does not contradict the general logic of the system’s operation and takes place in the usual cycle.
Using a live table reduced query response time from 7 minutes to a few milliseconds, because now you don’t need to iterate through millions of related records to process data. This data is quite sufficient for the active work of the company, and in addition, there remains a large amount of resources for expanding the table in the future. This is due to the peculiarity of the business, since the client works with “hot” leads, the “usefulness” of which is approximately one year.
Again, we have to return to the statement that the key to the implementation of the project was a detailed study of the client’s business processes. The “Reserve” stores data on leads with which no action is currently planned. These people may re-apply to the company, mainly with payment for services. In this case, custom APIs intercept payment notifications and check with the “Reserve” table. If the ice is there, it is transferred to the live table.
Results
As you can see, the solution from Bitrixoids really achieved two goals at once:
- Ensures an optimized process of processing requests without excessive load on the server;
- Stores the most valuable thing for CRM and business in general — customer information.
The massive database that has been formed since the first days of using Bitrix24 “rests” in storage and only occasionally responds to requests through the API or targeted requests from employees when they really need to find information about leads from many years ago.
It would be fair to say about the future additions to the task, namely the reverse transfer of data from the live table to the “Reserves” in the event of a change in the status of the lead (for example, “deleted”) or the expiration of the “expiry” period. However, for now, all entries in the table remain relevant and there is no urgent need for this. So if your Bitrix24 does not please you with speed due to the accumulation of information in the database or for other reasons, do not rush to delete something. Let Bitrixoids experts study your business processes and, we are sure, a custom technical solution will definitely be found!