Product Information Aggregator - IntexSoft

Product Information Aggregator

Aggregator of product information from various online stores.


The company’s main specialization is Data Science. Since the client had a lack of technical specialists, they turned to IntexSoft to create an aggregator for collecting product information from various online stores.


Team Composition

  • Back-end developer

  • Front-end developer


IT experts


months project duration


crawlers and scrapers updated


The main goal was to improve the existing prototype of the framework. Eventually, the prototype was completely redeveloped. Our team used Selenium to improve the collection of information from SPA (single-page application) stores.


IntexSoft specialists have added an API to force the product information update and display status information about crawlers/scrapers.


We managed to get the most out of the framework by scaling through processes and using proxies.


It was essential to implement the storage of all collected information in Azure Blob Storage, which was done successfully. Besides, IntexSoft specialists have added telemetry to Azure App Insights.


While working on the project, 38 crawlers and scrapers of online stores were updated and implemented.

Methodology & Transparency

The work was carried out using an iterative methodology. We arranged daily meetings with the client, where IntexSoft specialists demonstrated reports on completed tasks and resolved issues.


  • Information management

    collecting the information about products, saving information in various formats, a simple dashboard with information about system operation

  • Integrations

    integration of Azure Blob Storage and Azure Data Factory to move information from files to Blob Storage, integration of API

  • Launching

    different launch methods (link collection, product information collection), launch in a Docker container

Need a similar app? We can help!

Drop us a line