Spark-powered bioactivity prediction: a comparison of machine learning approaches

Abstract

The arduous and expensive nature of drug discovery has long been a bottleneck in scientific progress. However, recent breakthroughs in computational power, notably machine learning (ML) and artificial intelligence (AI), are profoundly transforming the field. Automated machine learning (AutoML) presents itself as a significant advancement, streamlining model selection, and hyperparameter tuning. This study delves into the potential of AutoML to accelerate drug discovery by comparing it to classical ML techniques. The focus lies on predicting the bioactivity of epidermal growth factor receptor (EGFR), a critical protein implicated in many cancers. By utilizing the scalability of Apache Spark, vast and diverse datasets encompassing biological, chemical, and genomic data tied to EGFR are processed. This comparative analysis aims to evaluate the comparative performance of both approaches, thereby contributing actionable insights to drug discovery research.