Algoworks helped an American based Texas Billing Platform plan and build a completely cloud bas[...]
Consultancy And Business Intelligence Services For Financial Firm Help Boosts Business Expansion Plans
Using Pentaho Business Analytics tools, Algoworks provided Consultancy, Support and Business Intelligence(BI) Services for an American based Billing Firm to help them with their optimization phase of the Growth Plan 2010-2015
Our client was a billion dollar finance and accounting firm with a huge US based client base. To increase their business to global multinationals, the client approached Algoworks to help them develop a huge multi-tenant based Software-as-a-Service product that they can sell to their global clients. We developed the product and it was a great success. Originally the client had their own legacy business intelligence solution that allowed them to analyze their's as well as their customer’s transactions using their SaaS platform’s databases. However their solution does not allow the customers to use the client’s business intelligence solution and were totally dependent on the client to create and process intelligence reports for them. This was proving to be hindrance to all-cloud-based image of the service. The client once again approached Algoworks to develop and integrate a robust business intelligence solutions to their SaaS platform that can be accessed by the end customers as well as the client. The new solution should be fast, secure, flexible, and have capabilities to handle a multi-tenant architecture. Based on these requirements and considering the fact that the platform was built on Java based technologies, we suggested using Pentaho Business Intelligence server and related tools for creating the BI solution.
How We Did It
- Secure Data Warehousing
- Pentaho BI Server Clustering
- Standard Report Generation
- Metamodel Ad Hoc Reports
- Data Cubes and Visual Reporting
Secure Data Warehousing
Challenge: How to process raw data without effecting performance of the main server
The main problem of legacy business intelligence system was its performance. The system uses data from the server, processes it and then uploads the processed data into reports. However every time a report needs to be generated, this process is repeated. The processing of raw data consumes valuable resources. The new solution should address these issues as well as create features to allow the multi-tenant SaaS platform to process data for each individual client.
Solution: Data warehousing for Multi-Tenant Architecture SaaS platform
Unlike the legacy system, we created a new data warehouse that processes the raw data and then stores it in its processed form. The main database and the new data warehouse was connected through an ETL based architecture utilizing Pentaho Data Integration tools. We automated the updation of data using Crontab based jobs that were executed in scheduled batches. However the client and the tenants of the client had features that allowed them to sync the database by a click of the button. In addition we created a custom load balancing and clustering program to integrate multiple Pentaho Servers, a facility not available by default in Pentaho. This allowed the client to scale up their operations and connect their global servers.
Pentaho BI Server Clustering
Challenge: How to scale the analytics solution to include multiple server
The client had a global operation with multi-billion dollar firms as end clients. To cater to global clients they created large hardware network using multiple servers for their SaaS based platform. As the operation of the client was increasing, the client needed a way to integrate these servers to balance loads equally for the optimum overall performance
Solution: A JAVA load balancing program to balance server load
Pentaho server dont have any in-built load balancing and clustering features. To connect global servers of the client we created a custom JAVA based load balancing program. The program was smart enough to balance the data fetching and data processing load on multiple servers based on their availability. The load balancer was also smart enough to balance the processes on servers based on their geographical locations to factor out location specific delays. This allowed the client to connect their multiple servers to single coherent solution and scale up their operations to target more global localities.
Standard Report Generation
Challenge: How to assemble the processed data into reports
The client needed three different types of reports. A timely standard reports, ad hoc custom reports, and cube based tabular and visual custom reports. The client needed a solution that can process the data to create fixed format reports in a weekly, monthly, or based on custom defined duration. These reports needed to be automatically dispatched to the related actors and to be made available in the actor’s SaaS dashboard as well as Pentaho dashboard.
Solution: Standard report generation using Pentaho Report Designer
We customized our Pentaho installation to create timely reports using processed data stored in previously mentioned data warehouse. The report generation process was handled by Pentaho Report Designer tool and was dispatched using the SaaS platform’s automated reporting tools. The client can customize the report dispatch timings on weekly, monthly, or daily basis, or even set their own dispatch frequency. Both end-clients and clients can customize their standards reports and can set reports’ data fields, selection of the data in the report, and report dispatch timings for the standard reports so that they can receive reports most relevant to their business. The client can receive the reports in multiple formats including XML, CSV, PDF, XLS and other database formats, and view the reports in tabular and visual format in their dashboard. The report generation process was strictly role based and each actor received their role specific reports. In addition the whole solution was multi-tenant based and single data warehouse was used for all end-clients with tested secure access.
Metamodel Ad Hoc Reports
Challenge: How to create custom reports with select data fields
Though standard reports are necessary, they sometimes fail to represent the exact business needs for specific actors. The users needed a feature that allowed them to create reports in ad hoc basis. The solution should have the capability to allow actors to create reports containing required fields. Once again security and role based restriction was paramount requirements and the solution should support the multi-tenant architecture.
Solution: Ad Hoc report generation using Pentaho Report Designer and Pentaho Metadata Editor
We created an ad hoc custom report generation tool using Pentaho Report Designer and Pentaho Metadata Editor tools. The tool had drag and drop features allowing user to simply drag-and-drop the necessary fields from a database explorer view bar, and create the reports via a simple click of the button. We divided our data warehouse into selective data marts that expedited report generation. The tool only displayed the data fields that a particular actor had access to, and only populated it with data values that the actor is allowed to view. In addition to customizing which data will appear at which column, the actors can filter the fetched data by as many filter levels they like and can group the data by any value they like, all using simple drag and drop features.
Data Cubes and Visual Reporting
Challenge: How to represent three dimensional level data
In today’s age of big data analysis and representation, two dimensional data is no longer sufficient. The clients needed a high performing three dimensional data report generation tool. These new types of data representation methods, called Cubes, are very performance intensive and require great understanding of database for perfect representation. However they are very useful in representing financial data in terms of three values like for example representing area wise product consumption in different time periods.
Solution: A drag and drop OLAP cube generation tool
Using the principles of our ad hoc report generation tool, we created a custom drag-and-drop cube generation tool. The three dimensional Online Analytical Processing (OLAP) data cubes were performance intensive but since we had already processed our data using separate data warehouse, the processing time was greatly reduced. The tool was very user-friendly with drag and drop features and require no coding from user’s end to fetch complicatedly filtered data. We also extended the features to represent the data in multiple visual formats such as pie charts, bar graphs, line graphs etc.
Report Generation Time Reduced By 30 Seconds
Our high performance tool now generates reports 30 to 90 seconds faster than the older solution, saving hours of time for analysts.
1.5+ Million Reports Processed Monthly
The analytic solution automatically dispatches 1500 different reports per tenant monthly and ~600 different reports weekly.
Analytics Solution Alone Makes Thousands
Our analytics solution was so robust that our client can now market the analytics solution itself as value added service
Clustered 20+ Servers To Give A Seamless Solution
We connected and clustered 20+ servers to present end-clients a seamless analytics experience.
We have been working with Algoworks for about a year now. They are a true partner that treats our projects like their own. We have engaged them on many simultaneous projects and they always provide us with clear and concise communication and a solid project plan. We know that when we involve Algoworks in a project it will be completed on time and within budget.
Joshua Kornitsky CTO at FRONTLINE Selling
Team Algoworks consistently went above and beyond in terms of responsiveness, adaptability to new requirements, and dedication to client and concept.
Bonnie Worthington CRM Analyst at Construction Specialties, Inc
These guys were great and handled everything perfectly. What I was most impressed with was their communication. They were constantly updating me on their progress... daily. Which really helps when you're a business owner. I highly recommend Algoworks.
Eric Pierce Founder, Clearview Elite | Lead Response Expert
They’re experienced to understand the pain points and their innovative solutions surely worked for me.
George Kirk VP Engineering at Innovazion
I like very much about the app development, I am happy with the services.
David Mortlock Principal/Lead Consultant at ComputaCenter
Checkout these related project to know about our company, our work, and our expertise.
Algoworks engineered the intranet HRM portal with Microsoft 365 Integration for a Multinational[...]
Algoworks provided Alfresco and Liferay Based LMMS (Learning Material Management Solution) To C[...]
For a firm famous for its workflow management and business communication services, we created a[...]
Are you facing similar problem in your project or if you want a custom solution fitting your needs, dont hesitate to contact us for a free consult and quote. Get our expert advice before moving forward.