Tom
4p3 comments posted · 0 followers · following 0
15 years ago @ Tom Pitts Dot Org - Omniture Summit 2010 R... · 0 replies · +2 points
15 years ago @ PeteSearch - http://petewarden.type... · 0 replies · +1 points
15 years ago @ PeteSearch - http://petewarden.type... · 0 replies · +1 points
The problem is definitely intent, but even if there was a 'no distribution' flag that prevents everything besides excerpts (much like fair use law for media), i think a lot of the sites would still be upset about various reasons crawlers are indexing their content. for instance in ecommerce there are large companies crawling competitors to index prices.