Chartable's crawler powers many of our services, including:

• Fetching new episodes for users of Chartable analytics, SmartLinks, SmartPromos, SmartAds and similar products
• Analyzing episode header information (file size, bit rate, ID3 tag size) so we can maintain download counts in accordance with our IAB Tech lab certification
• Maintaining our database of podcast and episode metadata

Our useragent for feed fetching is:

Mozilla/5.0 +https://chartable.com/crawler Trackable/0.1

Our useragent for episode fetching is unfortunately less descriptive, but we are maintaining it because we know hosting providers have accounted for it already:

Trackable/0.1

For feeds, we have a variable fetching frequency; for users of our Trackable analytics prefix, we may fetch as often as every 5-10 minutes to ensure that we can count downloads for new episodes as quickly as possible. Other feeds are fetched depending on their update frequency.

We cache heavily and make use of ETag and last-modified headers to reduce load on your feed server/generator.

For episode headers, we strive to only fetch each episode once. Depending on your host's URL structure, we may have to check files more than oince to ensure that different URLs map to the same MP3 file.

We have a standard timeout of 10 seconds and may retry occasionally.

If you have any concerns or questions related to our crawler, please email [email protected].