Big Data Analysis: Methodologies, Frameworks and Real-World Applications

Abstract

Inthelastyears,thecapacitytoproduceandcollectdatahasincreasedexpo- nentially.Thehugeamountofdatagenerated,commonlyreferredtoasBigData, thespeedatwhichitisproduced,anditsheterogeneityintermsofformatrepresent a challengetocurrentstorage,processing,andanalysiscapabilities.Thisscenario requiresthedesignandimplementationofnewarchitecturesandanalyticalplatform solutionsthatmustprocessBigDatatoextractcomplexpredictiveanddescriptive models.Today,high-performancecomputing(HPC)infrastructuressuchashighly parallelclusters,supercomputers,andcloudscanbeusedforprocessingandanalyz- ingmassivesourcesofreal-worlddatainvariousfields,includinggenomicsequencing andmedicalresearch,frauddetection,andweatherforecasting.Followingthesepre- liminaryobservations,thegoalofthisthesisistwofold.First,themainchallengesto besolvedforimplementinginnovativedataanalysisapplicationsonHPCsystemsare investigated.Inparticular,themainkeyresearchtopicsaddressedinclude:(i)stud- iesofsoftwaresystemsforBigDatastoring,processing,andanalysis;(ii)methods, techniques,andprototypesdesignedandusedtoimplementBigDatasolutionson massivedatasourcesrequiringtheuseofhigh-performancecomputingsystems;and (iii)designandprogrammingissuesforBigDataanalysisinExascalesystems,which willrepresentthenextcomputingstep.Second,severalinnovativeapplicationsand usecasesofBigDataanalyticsthatcanbeimplementedinlarge-scaleparallelsys- temsareproposed.Theseresearchcontributionsprovidenewinsightsandsolutions forextractingusefulknowledgefromlargevolumesofdata,describingmethodsand mechanismstosupportusers,practitioners,andscientistsworkingintheareaofBig Datainthedesignandexecutionofdataanalysistechniquesindifferentapplication domains.