Saving time and memory in computational intelligence system with machine unification and task spooling

摘要

There are many knowledge-based data mining frameworks and it is common to think that new ones cannot come up with anything new. This article refutes such claims. We propose a sophisticated unification mechanism and two-tier machine cache system aimed at saving time and memory. No machine is run twice. Instead, machines are reused wherever they are repeatedly requested (regardless of request context). We also present an exceptional task spooler. Its unique design facilitates efficient automated management of large numbers of tasks with natural adjustment to available computational resources. Dedicated task scheduler cooperates with machine unification mechanism to save time and space. The solutions are possible thanks to very general and universal design of machine, configuration, machine context, unique machine life cycle, machine information exchange, configuration templates and other necessary concepts. Results gained by machines are stored in a uniform way, facilitating easy results exploration and collection by means of a special query system and versatile analysis with series transformations. No knowledge about internals of particular machines is necessary to extensively explore the results. The ideas presented here, have been implemented and verified inside Intemi framework for data mining and meta-learning tasks. They are general engine-level mechanisms that may be fruitful in all aspects of data analysis, all applications of knowledge-based data mining, computational intelligence, machine learning or neural networks methods.