The past, present, and future of robots.txt
Listen now
Description
In this episode of Search Off the Record, Gary, Lizzi, and Martin from the Google Search team are joined by special guest David Price, a Product Counsel for Google. They discuss human initiated crawl requests vs. bot initiated crawl requests, the robots.txt file, and more! Episode transcript → https://goo.gle/sotr028-transcript Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
More Episodes
In this episode of Search Off the Record, John Mueller and Lizzi Sassman are joined by Rick Viscomi from the Chrome Web Performance team to discuss the importance of understanding what users seek to accomplish and optimizing accordingly, rather than focusing solely on metric scores. Listen along...
Published 03/28/24
Published 03/28/24