Deployment of temporal GNNs for mining large social networks over Spark on Kubernetes
Loading...
Date
2023-09-26
Authors
Σοφούλης, Φώτης
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This master thesis explores the field of graph neural networks and their applications in analyzing large social networks for node feature prediction. The objective of the thesis is twofold: firstly, leveraging the capabilities of Apache Spark on a Kubernetes-managed cluster, the research accentuates the significance of deploying such implementations on physical clusters. This approach sheds light on the inherent challenges posed by real network infrastructure, like network delays, synchronization discrepancies, and slow propagation of critical data. Secondly, the study focuses on the applicability of temporal graph neural networks, in capturing dynamic behaviors within large social graphs, such as label prediction and community detection. Through a structured approach, the research spans the foundations of machine learning concepts, delves into graph neural networks, elaborates on large social network analysis with Spark, and presents an overview of modern infrastructure trends. The thesis concludes by presenting a detailed exploration of the proposed infrastructure and the experimental results.
Description
Keywords
Temporal graph neural networks, Apache Spark, Kubernetes, Continuous deployment, PySpark, Power iteration clustering