Tesi di Dottorato
Permanent URI for this communityhttps://lisa.unical.it/handle/10955/10
Browse
3 results
Search Results
Item Distributed Optimization Over Large-Scale Systems for Big Data Analytics(Università della Calabria, 2020-01-17) Shahbazian, Reza; Leone, Nicola; Grandinetti, Lucio; Guerriero, FrancescaA large-scale system is defined as one that supports multiple, simultaneous users who access the core functionality through some network. Nowadays, enormous amount of data is continually generated at unprecedented and ever-increasing scales. Large-scale data sets are collected and studied in numerous domains, from engineering sciences to social networks, commerce, bimolecular research, and security. Big Data is a term applied to data sets whose size or type is beyond the ability of traditional relational databases to capture, manage, and process with acceptable latency. Usually, Big Data has one or more of the characteristics including high volume, high velocity, or high variety. Big Data challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy and data source. Generally, Big Data comes from sensors, devices, video or audio, networks, log files, transactional applications, web, and social media, in a very large-scale. Big Data is impossible to analyze by using traditional central methods and therefore, new distributed models and algorithms are needed to process the data. In this thesis, we focus on optimization algorithms for Big Data application. We review some of the recent machine learning, convex and non-convex, heuristic and stochastic optimization techniques and available tools applied to Big Data. We also propose a new distributed and decentralized stochastic algorithm for Big Data analytics. Our proposed algorithm is fully distributed to decide large-scale networks and data sets. The proposed method is scalable to any network configuration, is near real-time (in each iteration, a solution is provided although it might not be the optimum one) and more critical, robust to any missing data or communication failures. We evaluate the proposed method by a practical example and simulations on cognitive radio networks. Simulation results confirmed that the proposed method is efficient in terms of accuracy and robustness. We assume that the distributed data-sources should be capable of processing their data and communicate with neighbor sources to find the network objective as an optimal decision. Some challenges are introduced by new technologies such as 5G or high-speed wireless data transfer, including imperfect communications that damage the data. We propose an optimal algorithm that uses optimal weighting to combine the shared data coming from neighbors. This optimal weight improves the performance of the decision-making algorithm in terms of error and convergence rate. We evaluate the performance of the proposed algorithm mathematically and introduce the step-sized conditions that guaranteed the convergence of the proposed algorithm. We use computer simulations to evaluate the network error. We prove that in a network diagram with ten datasources, the network performance of the proposed algorithm outperforms some of the known optimal solutions such as Metropolis and adaptive combination. Keywords: Optimization, Big Data, Large-Scale, Distributed, Optimal Weight.Item Methodologies and Applications for Big Data Analytics(Università della Calabria, 2020-05-02) Cassavia, Nunziato; Crupi, Felice; Flesca, Sergio; Masciari, ElioDue to the emerging Big Data paradigm, driven by the increase availability of users generated data, traditional data management techniques are inadequate in many real life scenarios. The availability of huge amounts of data pertaining to user social interactions calls for advanced analysis strategies in order to extract meaningful information. Furthermore, heterogeneity and high speed of user generated data require suitable data storage and management and a huge amount of computing power. This dissertation presents a Big Data framework able to enhances user quest for information by exploiting previous knowledge about their social environment. Moreover an introduction to Big Data and NoSQL systems is provided and two basic architecture for Big Data analysis are presented. The framework that enhances user quest, leverages the extent of influence that the users are potentially subject to and the influence they may exert on other users. User influence spread, across the network, is dynamically computed as well to improve user search strategy by providing specific suggestions, represented as tailored faceted features. The approach is tested in an important application scenario such as tourist recommendation where several experiment have been performed to assess system scalability and data read/write efficiency. The study of this system and of advanced analysis on Big Data has shown the need for a huge computing power. To this end an high performance computing system named CoremunitiTM is presented. This system represents a P2P solution for solving complex works by using the idling computational resources of users connected to this network. Users help each other by asking the network computational resources when they face high computing demanding tasks. Differently from many proposals available for volunteer computing, users providing their resources are rewarded with tangible credits. This approach is tested in an interesting scenario as 3D rendering where its efficiency has been compared with "traditional" commercial solutions like cloud platforms and render farms showing shorter task completion times at low cost.Item Methodologies and Applications for Big Data Analytics(Università della Calabria, 2020-05-02) Cassavia, Nunziato; Crupi, Felice; Flesca, Sergio; Masciari, Elio;Due to the emerging Big Data paradigm, driven by the increase availability of users generated data, traditional data management techniques are inadequate in many real life scenarios. The availability of huge amounts of data pertaining to user social interactions calls for advanced analysis strategies in order to extract meaningful information. Furthermore, heterogeneity and high speed of user generated data require suitable data storage and management and a huge amount of computing power. This dissertation presents a Big Data framework able to enhances user quest for information by exploiting previous knowledge about their social environment. Moreover an introduction to Big Data and NoSQL systems is provided and two basic architecture for Big Data analysis are presented. The framework that enhances user quest, leverages the extent of influence that the users are potentially subject to and the influence they may exert on other users. User influence spread, across the network, is dynamically computed as well to improve user search strategy by providing specific suggestions, represented as tailored faceted features. The approach is tested in an important application scenario such as tourist recommendation where several experiment have been performed to assess system scalability and data read/write efficiency. The study of this system and of advanced analysis on Big Data has shown the need for a huge computing power. To this end an high performance computing system named CoremunitiTM is presented. This system represents a P2P solution for solving complex works by using the idling computational resources of users connected to this network. Users help each other by asking the network computational resources when they face high computing demanding tasks. Differently from many proposals available for volunteer computing, users providing their resources are rewarded with tangible credits. This approach is tested in an interesting scenario as 3D rendering where its efficiency has been compared with "traditional" commercial solutions like cloud platforms and render farms showing shorter task completion times at low cost.