Focused crawler based on concept context graph

Xiaolei Li; Yajun Du; Xiaoping Huang; Yufeng Hai

doi:10.1117/12.3012107

21 December 2023 Focused crawler based on concept context graph

Xiaolei Li, Yajun Du, Xiaoping Huang, Yufeng Hai

Proceedings Volume 12970, Fourth International Conference on Signal Processing and Computer Science (SPCS 2023); 1297034 (2023) https://doi.org/10.1117/12.3012107
Event: Fourth International Conference on Signal Processing and Computer Science (SPCS 2023), 2023, Guilin, China

Abstract

In order to improve the performance of focused crawler, a topic crawling strategy based on Concept Context Graph(CCG) is proposed. User knowledge background is constructed from the initial topic text returned by zhishi.me Knowledge Graph, and it is transformed into a CCG to guide the crawling of focused crawler. During the crawling process, the crawler treats the hyperlinks it encounters as a Virtual Concept(VC), and then calculates the Sememe Similarity(SS) between the VC and the core concept in CCG based on HowNet. According to the relationship between the SS value and the given threshold, it is decided whether to keep the link or not, and the CCG is updated for the reserved VC. The corresponding hyperlinks of the preserved VC are then stored in the queue to be crawled according to the order of SS value from high to low. Through experiments, it can be seen that the topic crawling strategy can improve the performance of focused crawler.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xiaolei Li, Yajun Du, Xiaoping Huang, and Yufeng Hai "Focused crawler based on concept context graph", Proc. SPIE 12970, Fourth International Conference on Signal Processing and Computer Science (SPCS 2023), 1297034 (21 December 2023); https://doi.org/10.1117/12.3012107

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Semantics

Internet

Mining

Software engineering

Statistical methods

Strategic intelligence

RELATED CONTENT

Message board information extraction based on LDA and RNN
Proceedings of SPIE (May 24 2022)

Web service recommendation based on knowledge graph enhanced
Proceedings of SPIE (May 25 2023)

Research on source code plagiarism detection based on multiple features
Proceedings of SPIE (July 19 2024)

Multi-level fast personalized recommendation system
Proceedings of SPIE (March 28 2023)

Intent prediction-based industrial data flow scheme
Proceedings of SPIE (June 28 2023)

A framework of Chinese semantic text mining based on ontology...
Proceedings of SPIE (January 13 2012)

Data mining for the e-business: developments and directions
Proceedings of SPIE (April 06 2000)

Subscribe to Digital Library

Receive Erratum Email Alert

Keywords/Phrases

Search In:

Publication Years