Software that automatically traverses web pages to collect their content and link structures. Also known as spiders or bots, web crawlers are used by full-text search engines to build their search indexes. The collected data is stored in an index that is queried when users submit search requests.

Past questions testing this term · 1 questions

• 2016h28a #86全文検索型検索エンジンの検索データベースを作成する際に用いられ，Web ページを自動的に巡回・収集するソフトウェアはどれか。