Data Mining Through Relational Database

Abstract

With the wide availability of huge amounts of data and the imminent demands to transform raw data into useful information and knowledge, data mining has become an important research field in both the database area and the machine learning area. Data and information (or knowledge) play a significant role in human activities. Data mining is the process of knowledge discovery by analyzing large volumes of data from various perspectives and summarizing it into useful information. Due to the importance of extracting knowledge/information from large data repositories, data mining has become an essential component in various fields of human life. Advancements in statistics, machine learning, artificial intelligence, pattern recognition, and computational capabilities have evolved modern-day data mining applications, which now enrich various domains such as business, education, medicine, and science. Hence, this paper discusses the various improvements in the field of data mining from past to present and explores future trends. Data mining is defined as the process of solving problems by analyzing data already present in the database and discovering knowledge within it. Database systems provide efficient data storage, fast access structures, and a wide variety of indexing methods to speed up data retrieval. Machine learning provides theoretical support for most of the popular data mining algorithms. The database approach combines the properties of both areas to improve the scalability of Weka, an open-source machine learning software package. Weka implements most machine learning algorithms using main-memory-based data structures, which means it cannot handle datasets larger than the available main memory. A database is implemented to store and access data from DB2, achieving better scalability than Weka. However, the database is slower than Weka because secondary storage access is more expensive than main memory access. In this thesis, we extend the database with a buffer management component to improve its performance. Furthermore, we increase scalability by incorporating additional data structures into the database, utilizing a buffer for data access. We also explore another method to improve algorithm speed by leveraging the data access properties of machine learning algorithms.