Data lake development status and outlook

Guonong Li; Wenyu Hu; tongzhen You

doi:10.1117/12.3011074

20 October 2023 Data lake development status and outlook

Guonong Li, Wenyu Hu, tongzhen You

Proceedings Volume 12814, Third International Conference on Green Communication, Network, and Internet of Things (CNIoT 2023); 128142G (2023) https://doi.org/10.1117/12.3011074
Event: Third International Conference on Green Communication, Network, and Internet of Things (CNIoT 2023), 2023, Chongqing, China

Abstract

To adapt to the increasing demand for big data storage and analysis, and to solve the difficulties and pain points of Data Warehouse technology, it is necessary to build an efficient and high-quality data storage architecture. In this thesis, the evolution, technical background, architecture, and the current status of domestic and international research of the Data Lake concept were presented, Data Lake and Data Warehouse were compared and their advantages and disadvantages were summarized. The feasibility of Data Lakehouse and ‘Data Lake + Data Middle-end’ was discussed, and the problems that should be solved in the construction of ‘Data Lake + Data Middle-end’ were prospected. Finally, the challenges faced by the further development of Data Lake technology were analyzed, and some ideas to solve these problems were put forward.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Guonong Li, Wenyu Hu, and tongzhen You "Data lake development status and outlook", Proc. SPIE 12814, Third International Conference on Green Communication, Network, and Internet of Things (CNIoT 2023), 128142G (20 October 2023); https://doi.org/10.1117/12.3011074

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available