Utilize este identificador para referenciar este registo:
https://hdl.handle.net/1822/72111
Título: | Catching web crawlers in the act |
Autor(es): | Lourenço, Anália Maria Garcia Belo, Orlando |
Palavras-chave: | Data webhousing Traverse patterns Web crawler Web usage mining |
Data: | 2006 |
Editora: | Association for Computing Machinery (ACM) |
Citação: | Anália G. Lourenço and Orlando O. Belo. 2006. Catching web crawlers in the act. In Proceedings of the 6th international conference on Web engineering (ICWE '06). Association for Computing Machinery, New York, NY, USA, 265–272. DOI: https://doi.org/10.1145/1145581.1145634 |
Resumo(s): | This paper recommends a new approach to the detection and containment of Web crawler traverses based on clickstream data mining. Timely detection prevents crawler abusive consumption of Web server resources and eventual site contents privacy or copyrights violation. Clickstream data differentiation ensures focused usage analysis, valuable both for regular users and crawler profiling. Our platform, named ClickTips, sustains a site-specific, updatable detection model that tags Web crawler traverses based on incremental Web session inspection and a decision model that assesses eventual containment. The goal is to deliver a model flexible enough to keep up with crawling continuous evolving and that is capable of detecting crawler presence as soon as possible. We use a real-world Web site case study as a support for process description, as well as, to evaluate the accuracy of the obtained classification models and their ability for discovering previously unknown Web crawlers. |
Tipo: | Artigo em ata de conferência |
URI: | https://hdl.handle.net/1822/72111 |
ISBN: | 1595933522 |
DOI: | 10.1145/1145581.1145634 |
Versão da editora: | https://doi.org/10.1145/1145581.1145634 |
Arbitragem científica: | yes |
Acesso: | Acesso restrito UMinho |
Aparece nas coleções: |
Ficheiros deste registo:
Ficheiro | Descrição | Tamanho | Formato | |
---|---|---|---|---|
2006-CI-ICWE-Lourenco&Belo-CRP.pdf Acesso restrito! | 781 kB | Adobe PDF | Ver/Abrir |