Data scientists are often faced with selecting the most efficient algorithms to solve a given business problem. This process can be very time-consuming if they do not already know what they are looking for. By combining domain expertise and Big Data Science techniques, the Scry-Jidoka platform provides a decision support system (DSS) to help users choose and execute appropriate algorithms in order to analyze their data and solve the problem. To achieve this, Scry-Jidoka’s DSS uses a rich library of statistical and mathematical operators to analyze the settings in which various algorithms were previously successful, thereby providing appropriate recommendations. Some of the highlighted features of Scry-Jidoka are given below.



To use Scry-Jidoka, users input the processed (cleansed and harmonized) data, via Scry-Collatio or otherwise, and use its Graphical User Interface (GUI) to visually understand it. Given this data and the specific business problem, this platform filters for useful algorithms that may be proprietary or Open Source and stored in its library. Through the GUI, the users can see the utility of these algorithms for descriptive and statistical analysis, and then test their hypothesis using any of these algorithms. Finally, based on the outcome, users can either select algorithms individually or use Scry-Jidoka’s DSS to combine them and achieve better results.



Using the given data, Scry-Jidoka helps users determine the parameters that are most relevant to the business problem. Based on these results, this platform predicts the dynamics of the key performance indicators (KPIs) for this problem. Finally, this platform provides prescriptive analysis and actionable insights as to how to improve these KPIs, thereby leading to enhanced performance.



The algorithms – both Open Source and proprietary – stored in Scry-Jidoka’s library are state-of-the-art. In case Open Source algorithms do not meet our needs, we create our own proprietary algorithms to solve specific problems; our algorithms incorporate concepts that are still actively under research by theoretical computer scientists including deep learning and hypergraph modeling. Furthermore, this platform contains many modified algorithms for analyzing unstructured data, audio and video data, and wave-forms data (e.g., ECG data).


Scry-Jidoka starts by ingesting different data-sets that may not reside in a single comprehensive database. This data is input either from the Scry-Collatio platform or from clients’ databases (that contain analysis-ready data) through Scry-Collatio’s data-connector tools and application program interfaces. In both situations, cleansing, munging and harmonizing of data has occurred before it is provided to the Scry-Jidoka platform.



Scry-Jidoka platform has pre-built libraries that are used for “exploring” data with respect to potential problems and noise; these software libraries help a user in imputing missing values, computing caps and floors, transforming data (as required for the business problem), and noise reduction. In addition, Scry-Jidoka contains several libraries containing statistical algorithms (e.g., generalized linear model, logistic regression, random forests, ARIMA, SARIMA, ETS, Kalman Filters, and related techniques) that help domain experts and data scientists in a deeper understanding of the harmonized data and in deriving actionable insights. For a given business problem, Scry-Jidoka’s decision support system (DSS) helps users choose and execute appropriate algorithms – as well as combine these algorithms if needed – so as to produce better insights. Intuition from data scientists and subject matter experts complements this entire process, thereby, giving this platform its name: automated computing with a human touch.



The machine learning, natural language processing and information retrieval libraries in Scry-Jidoka are designed to handle various steps, which include tokenizing, POS tagging, sentence splitting, entity recognition, relation extraction, feature extraction, topic modeling, similarity algorithms, k-nearest neighbor, state space modeling, and classification algorithms.  In addition, this platform contains our proprietary algorithms that are related to deep learning algorithms and recurrent neural networks, which have also been modified to work extremely well in specific domains (e.g., Oncology) and within specific industries.



For solving a specific problem, data scientists and subject matter experts work together to pick the appropriate algorithms (as suggested by Scry-Jidoka’s DSS or on their own), execute these algorithms on the given data, and obtain the final results. In fact, using Scry-Jidoka, these professionals can compare two or more solutions by executing two or more sets of algorithms on the same data set and decide as to which one works better for their use-case. Scry-Jidoka’s visualization provides Graphics User Interface (UI) functionalities so that outputs can be visualized by concurrent users. Indeed, these outputs can be depicted in different kinds of pie-charts, bar-charts, two dimensional graphs, graphs with nodes and edges that depict dataset mapping, data quality distribution, and various object relationships.