Mobility
Obtaining valuable information from vast amounts of data and analysing it is extremely demanding. For example, it is important to obtain results within a certain time frame. This requires the right tools and an experienced team. At the same time, there are unexpected challenges in the real world – you may have to revise your assumptions, e.g. And most importantly, your client tries to meet the deadlines. In short, from time to time you may have to process much more data in less time. That’s why scaling is so important – allocating and using much more calculation resources efficiently in less time.
The mobility data provider
CE-Traffic is a provider of traffic and mobility information services for private businesses and the public sector. We use anonymised location data from hundreds of thousand connected GPS devices to monitor and analyse road traffic information. They also uses anonymised signalling data from the mobile operator network for monitoring and analysis of people being present in certain locations or travelling e.g. counting visitors of selected sites, advanced tourism statistics, origin-destination analysis.
MELODIC supports in one of the basic challenges of working on data analysis
MELODIC helps to automatically scale data processing so that the results are always on time
The journey
What is necessary for both road and city authorities and business owners? Historical traffic information. Depending on the application, customer requirements and the scope of analysis of historical traffic information vary considerably. From a long-term analysis of the city’s core network or even an analysis of the nationwide motorway network to the assessment of a single point on a selected road section. At the same time, sometimes the historical data is not sufficient and a simulated environment is required for the “if what” analysis. CeTraffic decided to apply this scenario using the capabilities of the MELODIC platform.
CeTraffic needs an application ready for automatic deployment, which can be scaled horizontally. A two-component application consisting of a process control manager and Simulation Worker (with multiple instances) has been developed, using open source technologies such as Python, Celery and Redis.
It then defined what information MELODIC must monitor in order to find the best implementation plan. The following set was finally defined:
the remaining duration of the experiment,
the remaining number of simulations,
the average duration of a single simulation.
Based on these metrics, MELODIC can continuously calculate the minimum number of cores required to complete the experiment on time and thus decide on the size and number of machines in operation.
The application has been modelled in Camel and a utility function has been defined which minimizes implementation costs. Thanks to MELODIC, automatic implementation of the simulation experiment in the cloud is possible with one click.
The results
Until now, all CeTraffic applications have been implemented manually or semi-automatically. Based on their experience, people decided on the required resources, e.g. size and number of virtual machines. This is a time-consuming process, prone to human error. Moreover, adaptation usually requires human action or at least is limited to a pre-defined scenario.
With MELODIC you can now experiment effortlessly with different settings and with a very different number of simulations per experiment. By expressing and enforcing constraints in real time, the results are always delivered on time, whether hundreds or thousands of simulations are required or whether a single simulation takes 10 seconds or 10 minutes. MELODIC continuously monitors the implementation of the application and automatically allocates the required resources without human intervention.
This has a significant impact on data analyst performance. They can now focus on interpreting and understanding the data, rather than providing the resources needed to conduct their experiments. CeTrafric will also perform other scenarios to make the most of MELODIC.
Want to know more?
See CE-Traffic site www.ce-traffic.com »
More use cases
See how Renderro uses MELODIC to enable video and image creation in the Cloud.