I got into a very heated discussion today with some editors and manager types over at Urban Vancouver.
The site is an aggregator of blogs from across Vancouver and the web. They access the RSS feeds distributed by the blog and repost the entire blog entry on their site. I call it scraping, they call it aggregating.
Some tried to defend what they do by likening it to a mashup. However, mashups, as they are commonly found on the web and in music today, take an original work, combine it with another and make an entirely new product.
Taking my copy, republishing it on your website, with a unique url to your website, and linking back to the images from my server is not an entirely new product. It’s theft. It’s plagiarism. It’s wrong.
It’s not just my content that they’re stealing with urls unique to Urban Vancouver, they’ve aggregated dozens of blogs accross the city. Taking the content verbatim, plagiarizing it at an Urban Vancouver url.
Some are fine with it, I am not. Mostly because I wasn’t asked. I don’t need to be asked if you want to quote and link back to my blog. I don’t need to be asked if you want to blockquote and link back to my blog. Those sorts of tactics help evolve discussion on the web and flow traffic through varying thoughts and points of view.
Taking my entire blog entry, however, and reposting it on your site with an url unique to your domain, along with a second link that goes to my archive – on your site – is wrong.
I use FeedEntryHeader plugin for WordPress to try and prevent this sort of thing, or at least make it glaringly obvious when it happens. My header had read:
I’m wondering if I need to make it even stronger or more blatant.
Credit to the people at Bryght / Raincity, they did remove the stolen blog content the minute I called and asked for it to be done, but Roland‘s smug chuckle on the other end of the phone just told me these people know they’re getting one over on the bloggers of the city and frankly don’t give a damn.
Actually, it might not be the bloggers they’re getting over on, it’s more likely the rest of the PR sphere that is getting gamed. Boris Mann defends his site’s approach by saying: “Many people benefit from the ability of UV to help generate “press” credentials, for everything from music awards to the Olympics.”[source]
In other words, a few bloggers, without much clout, readership, or influence, are trying to artificially increase their standings on the internet so they can catch some creds and go to a free concert.
It’s cool to quote me (and link back to the source). It’s cool to blockquote me (and link back to the source). It’s not cool to repost all of my content with a unique URL that out Googles me when people are searching for keywords relating to my own content.
You can track through the comments on Twitter to try and rebuild the conversation. But here’s how most of it went down:
sillygwailo: @buzzbishop I’m a managing editor of Urban Vancouver, mostly adding feeds to the aggregator (and fighting spam comments/posts) these days.
about 7 hours ago · Reply · View Tweet
sillygwailo: @buzzbishop Got an example of the wrong URL? Every post links back to the original.
less than a minute later · Reply · View Tweet
buzzbishop: @sillygwailo every post also contains EACH and EVERY word and EACH and EVERY image in the original. so what’s the point of clicking back?
1 minute later · Reply · View Tweet
sillygwailo: @buzzbishop true enough. I’m asking about “does not drive to the original url”. It does indeed drive traffic to the original URL.
3 minutes later · Reply · View Tweet
buzzbishop: @sillygwailo cant demonstrate anymore as i’ve been removed from the site. here’s my issue: http://tinyurl.com/6k9trr (expand)
1 minute later · Reply · View Tweet
sillygwailo: @buzzbishop That UV reposts full content is undeniable. We wish the technology was a little more flexible in doing just excerpts or links.
less than 10 seconds later · Reply · View Tweet
buzzbishop: @sillygwailo how about you just leave peoples stuff alone til you figure out?
less than a minute later · Reply · View Tweet
sillygwailo: @buzzbishop (Not sure I understand the importance of the link you dropped. Wrong link?)
2 minutes later · Reply · View Tweet
sillygwailo: @buzzbishop We’ve been trying to find the balance of increasing the exposure of the Vancouver blogosphere and respecting rights.
1 minute later · Reply · View Tweet
buzzbishop: @sillygwailo no, it showed that your page out googled mine, taking traffic. and since my posts were FULLY republished without permission
less than 20 seconds later · Reply · View Tweet
buzzbishop: @sillygwailo there was no need for people to come to my page, cause you had (stolen) reposted my entire site’s content.
half a minute later · Reply · View Tweet
sillygwailo: @buzzbishop I’m open to the idea that the site swung the balance too far. Some people like it….
less than 5 seconds later · Reply · View Tweet
sillygwailo: @buzzbishop …the aggregator is used as source material for a certain daily Vancouver news wrap-up (or if it’s not now, was for some time).
1 minute later · Reply · View Tweet
buzzbishop: @sillygwailo here’s the thing.. you should ASK before you steal. then its not stealing. if people like it they will participate.
less than a minute later · Reply · View Tweet
sillygwailo: @buzzbishop The @bryght account doesn’t accept DMs because that account doesn’t follow anybody (I’m its most frequent updater, obviously).
2 minutes later · Reply · View Tweet
sillygwailo: @buzzbishop I’ve agonized over that. We’re different from scrapers because there’s no ‘nofollow’ and respond to removal requests.
2 minutes later · Reply · View Tweet
buzzbishop: @sillygwailo you act first. wait for people to notice later. you’re being sneaky. if it’s so great, so wonderful, then ASK. why be sneaky?
2 minutes later · Reply · View Tweet
sillygwailo: @buzzbishop Since January of this year, I’ve announced every addition at http://jaiku.com/channel/urbanvancouver
4 minutes later · Reply · View Tweet
buzzbishop: @sillygwailo its not the same. fck me you are just not getting it. ASK! 3 little letters A! S! K!.. not to be confused with A! S! S! (holes)
3 minutes later · Reply · View Tweet
buzzbishop: @sillygwailo negative option billing didnt work for rogers. stealing content that way wont work for you http://www.ellenroseman.com/?p=22
3 minutes later · Reply · View Tweet
buzzbishop: @sillygwailo i dont care if youre blogging for mother theresa. ASK before you repost my ENTIRE BLOG ARCHIVE.
1 minute later · Reply · View Tweet
sillygwailo: @buzzbishop Fair enough. I quibble that it’s not your entire blog archive: it expires posts. (I really don’t want the quibble to detract.)
6 minutes later · Reply · View Tweet
sillygwailo: @buzzbishop I agree with you.
20 minutes later · Reply · View Tweet
buzzbishop: @sillygwailo guilt would be a big red flag you are acting without ethics.
1 minute later · Reply · View Tweet
sillygwailo: @buzzbishop I don’t dispute that either. Surely you’ve agonized over stuff you’ve done?
7 minutes later · Reply · View Tweet [source]
I am not happy with the emotional tone of my tweets. I would have liked to have remained calm, but faced with stonewalled responses and a lack of addressing my original issue of their refusal to ask permission before wholesalely reposting/scraping/plagiarizing/stealing my content, I got a little emotional – it’s what I do.
I would love to have posted screen shots and direct links to the violations, but I didnt take them and the content has been removed. You can go to the site on your own to see how they’re scraping the content from the dozens of other bloggers around the city, I’d just rather not give them any direct links to validate their business practices.
“It does indeed drive traffic to the original URL.”
I use RSS readers to read blogs, and if Urban Vancouver has someone else’s entire article on their feed there is probably very little click-through to the original page.
What kind of referral traffic have you had from UV, if any? I believe they also have my blog up there (though it’s less of a deal to me at this point).
I noticed my food blog on Urban Vancouver 2 months ago, when I did a vanity search on Google and found one of my posts, with an Urban Vancouver URL, in the #9 spot. Promptly asked them to remove Tiny Bites and my personal blog (which I never hope to see syndicated again)–a request they accommodated straight away.
Before I sent the request over, I checked my web stats to determine whether I was actually getting referral traffic from the site: nope.
@david no, i dont get any traffic from UV. i do suspect, however, they get quite a bit of traffic from gaming other people’s content. in essence they siphon off your traffic to their site as all the keywords, links, and seo you assign to your post magically get assigned to their site as well.
Depending on where they are taking your feed from I guess depends on what you can do physically to stop people from scraping your feed.
If they are taking from your domain feed (http://yourdomain.com/feed) or something similar you can use an .htaccess file to block various IPs/Domains etc.
If they are taking it from say Feedburner, not sure what you could with that. Not sure if Feedburner has an option to block people from reading your feed, they should if they don’t.
(their website is currently timing out on me so I can’t see it lol)
Oh I forgot to mention, creating an excerpt is really not hard to do. If you have programmers that are smart enough (lol) you can easily substring the full feed to like 100, 150, 200 or whatever amount of characters and then just hyperlink that back to the original source.
I have a site that I run that reports on other sites content. I don’t scrap them, I manually read over their content and create my own posts and then quote a small portion with a link back to their original article.
UV has been running since April 2004. There was nothing else like it (pretty much anywhere) at the time. It was part technology demo, part community research project.
It is meant as an aggregator of all Vancouver content. People can come to one place and (through a browser) read a ton of content. And of course, people can post events, blog, etc. — not so important today when that functionality is available everywhere, but back in the day, there were few places you could easily / cheaply do this.
The links in the right sidebar? Recent posts, they all go straight to the blogs in question.
Want to read via RSS? Great, go the sources page (http://urbanvancouver.com/aggregator/sources) scroll to the very bottom, and use the OPML file to get a list of all blogs. Drink directly from the Vancouver blog fire hose. Which, of course, links to the original blogs.
People ask to be included. Others ask to be removed. We think it provides value, and our intent is not to steal or do anything malicious.
Commercial? I think it makes about $2 / month on adsense (I’d have to check .. it’s essentially negligible).
“Roland’s smug chuckle” — anyone that knows Roland knows that he is ANYTHING but smug.
@boris the solution is simple: ASK before you re-appropriate someone else’s content.
have it be opt in. send a notice to each and every author you currently scrape, notifying them of what you’re doing and give them the option to be included or not.
unilaterally reposting content without notifying the original authors can be anything from copyright violation, to plagiarism to simply inconsiderate.
if your team is adding feeds manually, they have the time and energy to seek permission from the authors first.
if this technology is well behind the norm, remove the site. build something else.
as for roland, i have heard anecdotal evidence from other bloggers who have asked to be removed that he was less than poilte about it. arrogant, smug, cocky, indifferent – take your pick.
I was a small part of some of the discussions yesterday, mainly because this is an area (especially with regards to photography) that I have a vested interested in. I’ll also go on the record and state that the guys at Urban Vancouver have helped me get into a few high profile events based on the traffic of Urban Vancouver, which was much appreciated. That said, I was always under the impression that all the content on UV was opt-in, not opt-out. But given what i know about Boris, Roland and everyone else involved, I don’t think there’s anything malicious or devious going on (at least with regards to intention). But, I think in in the future the perception might be better if it is moved to opt-in, or as you suggested, sending a polite notice to someone when their blog is included. Just my 0.02.
@duane thanks so much for weighing in. i agree with it all.
my main issue has been, and will remain, the negative option they use for the content. the refusal to properly explain their actions, or explain why they act without permission is what frustrated me to no end yesterday and continues to bother me today.
they may provide a useful and valuable service for those who need seo help, but they shouldnt plunder through the feeds of every blogger in the city and repost the content at their own whim.
hell, even the spambot comment pages and link farms i see still only excerpt content when they scrape it.
the full scale reposting of entire blog entries including art and images without permission is the issue.
I, for one, am okay with the fact that Urban Vancouver aggregates my site content.
I was very careful to set up my Creative Commons license in this way, so that other sites could use my content in exchange for full attribution with link. When the site linking to me outranks mine, and my words and pictures are exposed to that bigger audience (not to mention the SEO boost from the link), I consider it a win.
In this regard, I am not alone. It has become commonplace to license content in this way, and Richard is guilty of having missed the variation in Buzz’s license. It’s likely that this has happened more than once, but never with any malicious intent, I’m sure.
Buzz, I almost understand why you prefer to stunt the growth of your blog by remaining a human gatekeeper of the use of your content, and it’s easy to see how you could be so incensed by the misuse of your content, in violation of your license. But your tone throughout has been less than gallant. You could have handled this calmly and actually educated a lot of folks on fair use and started a lively debate.
But you dared to imply that Roland was anything but a gentleman, and I here I am on the record in his defense. While I wasn’t there, I’d bet my hat that his chuckle was meant to try to disarm you in his almost trademarked jovial way, and you misinterpreted it through your seething rage.
i agree my tone has been less than cordial, that can happen when you’re trying to have a problem solved, and those who created the problem refuse to acknowledge the problem.
i stand by my impression of the phone conversation, i appreciate others have a different impression for knowing the man personally, i’m speaking to a 2 min conversation i had where my request was treated as frivolous and my opinion of the situation deemed ignorant.
THAT is what elevated my temper. the simple fact that there has been no admission of inappropriateness and that those who make the requests for removal are somehow ignorant to the system and a burden on their day.
I will leave the post up here so the comments and thoughts are on the record.
I just got off the phone with Kris from Bryght/Raincity. He’s a passionate guy about building and sharing and collaborating. Those in the community buy in to that process as well.
We had a great chat, and I’m sure the work that’s done by UV was with the best of intentions.
I, and others, kinda just wanna be asked before our stuff is fully republished.
Buzz, if you really want to prevent this sort of things from happening, I’m doing some research on copyright law in regard to bloggers. I’ll probably bring it up with everyone at Dot Com Pho in a few weeks.
I know our blogs get scraped. Mine does sometimes. Michael’s does too…and John is scraped by several scraper blogs. I’m looking into what we can do about it…since I find it annoying that someone takes our content.
[…] Some bloggers, notably in Vancouver, openly admit to this sort of gaming of the system. Boris Mann of Urban Vancouver has defended his site’s scraping and republishing of content by saying it was a means to help bloggers gain access to events. “Many people benefit from the ability of UV to help generate “press” credentials, for everything from music awards to the Olympics,” he said. […]
“It’s theft.”
Copyright infringement, not theft because nobody has been deprived of anything tangible. Although I agree with most of your post.
@Simon thanks. as I said, I’m a pretty emotional guy.
Legally it may be an infringement, personally it feels like a theft.
Thanks for the comment and clarification.
“It does indeed drive traffic to the original URL.”
I use RSS readers to read blogs, and if Urban Vancouver has someone else’s entire article on their feed there is probably very little click-through to the original page.
What kind of referral traffic have you had from UV, if any? I believe they also have my blog up there (though it’s less of a deal to me at this point).
I noticed my food blog on Urban Vancouver 2 months ago, when I did a vanity search on Google and found one of my posts, with an Urban Vancouver URL, in the #9 spot. Promptly asked them to remove Tiny Bites and my personal blog (which I never hope to see syndicated again)–a request they accommodated straight away.
Before I sent the request over, I checked my web stats to determine whether I was actually getting referral traffic from the site: nope.
@david no, i dont get any traffic from UV. i do suspect, however, they get quite a bit of traffic from gaming other people’s content. in essence they siphon off your traffic to their site as all the keywords, links, and seo you assign to your post magically get assigned to their site as well.
fun how that works, isnt it?
Depending on where they are taking your feed from I guess depends on what you can do physically to stop people from scraping your feed.
If they are taking from your domain feed (http://yourdomain.com/feed) or something similar you can use an .htaccess file to block various IPs/Domains etc.
If they are taking it from say Feedburner, not sure what you could with that. Not sure if Feedburner has an option to block people from reading your feed, they should if they don’t.
(their website is currently timing out on me so I can’t see it lol)
Oh I forgot to mention, creating an excerpt is really not hard to do. If you have programmers that are smart enough (lol) you can easily substring the full feed to like 100, 150, 200 or whatever amount of characters and then just hyperlink that back to the original source.
I have a site that I run that reports on other sites content. I don’t scrap them, I manually read over their content and create my own posts and then quote a small portion with a link back to their original article.
[…] Another Edit: Buzz has posted about it here. […]
I think intent matters here, Buzz.
UV has been running since April 2004. There was nothing else like it (pretty much anywhere) at the time. It was part technology demo, part community research project.
It is meant as an aggregator of all Vancouver content. People can come to one place and (through a browser) read a ton of content. And of course, people can post events, blog, etc. — not so important today when that functionality is available everywhere, but back in the day, there were few places you could easily / cheaply do this.
The links in the right sidebar? Recent posts, they all go straight to the blogs in question.
Want to read via RSS? Great, go the sources page (http://urbanvancouver.com/aggregator/sources) scroll to the very bottom, and use the OPML file to get a list of all blogs. Drink directly from the Vancouver blog fire hose. Which, of course, links to the original blogs.
People ask to be included. Others ask to be removed. We think it provides value, and our intent is not to steal or do anything malicious.
Commercial? I think it makes about $2 / month on adsense (I’d have to check .. it’s essentially negligible).
“Roland’s smug chuckle” — anyone that knows Roland knows that he is ANYTHING but smug.
Your blog got removed. What else would you like?.
@boris the solution is simple: ASK before you re-appropriate someone else’s content.
have it be opt in. send a notice to each and every author you currently scrape, notifying them of what you’re doing and give them the option to be included or not.
unilaterally reposting content without notifying the original authors can be anything from copyright violation, to plagiarism to simply inconsiderate.
if your team is adding feeds manually, they have the time and energy to seek permission from the authors first.
if this technology is well behind the norm, remove the site. build something else.
as for roland, i have heard anecdotal evidence from other bloggers who have asked to be removed that he was less than poilte about it. arrogant, smug, cocky, indifferent – take your pick.
I was a small part of some of the discussions yesterday, mainly because this is an area (especially with regards to photography) that I have a vested interested in. I’ll also go on the record and state that the guys at Urban Vancouver have helped me get into a few high profile events based on the traffic of Urban Vancouver, which was much appreciated. That said, I was always under the impression that all the content on UV was opt-in, not opt-out. But given what i know about Boris, Roland and everyone else involved, I don’t think there’s anything malicious or devious going on (at least with regards to intention). But, I think in in the future the perception might be better if it is moved to opt-in, or as you suggested, sending a polite notice to someone when their blog is included. Just my 0.02.
@duane thanks so much for weighing in. i agree with it all.
my main issue has been, and will remain, the negative option they use for the content. the refusal to properly explain their actions, or explain why they act without permission is what frustrated me to no end yesterday and continues to bother me today.
they may provide a useful and valuable service for those who need seo help, but they shouldnt plunder through the feeds of every blogger in the city and repost the content at their own whim.
hell, even the spambot comment pages and link farms i see still only excerpt content when they scrape it.
the full scale reposting of entire blog entries including art and images without permission is the issue.
I, for one, am okay with the fact that Urban Vancouver aggregates my site content.
I was very careful to set up my Creative Commons license in this way, so that other sites could use my content in exchange for full attribution with link. When the site linking to me outranks mine, and my words and pictures are exposed to that bigger audience (not to mention the SEO boost from the link), I consider it a win.
In this regard, I am not alone. It has become commonplace to license content in this way, and Richard is guilty of having missed the variation in Buzz’s license. It’s likely that this has happened more than once, but never with any malicious intent, I’m sure.
Buzz, I almost understand why you prefer to stunt the growth of your blog by remaining a human gatekeeper of the use of your content, and it’s easy to see how you could be so incensed by the misuse of your content, in violation of your license. But your tone throughout has been less than gallant. You could have handled this calmly and actually educated a lot of folks on fair use and started a lively debate.
But you dared to imply that Roland was anything but a gentleman, and I here I am on the record in his defense. While I wasn’t there, I’d bet my hat that his chuckle was meant to try to disarm you in his almost trademarked jovial way, and you misinterpreted it through your seething rage.
One other thing .. buzzbishop.com/blog was also being scraped by UV. right on the front page of buzzbishop.com there is a notice:
“all site content copyright cyberbuzz media. no unauthorized use permitted.”
authorization guys.. that’s what you need AUTHORIZATION
@jordan thanks for the comment.
i agree my tone has been less than cordial, that can happen when you’re trying to have a problem solved, and those who created the problem refuse to acknowledge the problem.
i stand by my impression of the phone conversation, i appreciate others have a different impression for knowing the man personally, i’m speaking to a 2 min conversation i had where my request was treated as frivolous and my opinion of the situation deemed ignorant.
THAT is what elevated my temper. the simple fact that there has been no admission of inappropriateness and that those who make the requests for removal are somehow ignorant to the system and a burden on their day.
it can be very frustrating.
I will leave the post up here so the comments and thoughts are on the record.
I just got off the phone with Kris from Bryght/Raincity. He’s a passionate guy about building and sharing and collaborating. Those in the community buy in to that process as well.
We had a great chat, and I’m sure the work that’s done by UV was with the best of intentions.
I, and others, kinda just wanna be asked before our stuff is fully republished.
And that’s that.
Way to stand up for your stuff.
Buzz, if you really want to prevent this sort of things from happening, I’m doing some research on copyright law in regard to bloggers. I’ll probably bring it up with everyone at Dot Com Pho in a few weeks.
I know our blogs get scraped. Mine does sometimes. Michael’s does too…and John is scraped by several scraper blogs. I’m looking into what we can do about it…since I find it annoying that someone takes our content.
[…] Some bloggers, notably in Vancouver, openly admit to this sort of gaming of the system. Boris Mann of Urban Vancouver has defended his site’s scraping and republishing of content by saying it was a means to help bloggers gain access to events. “Many people benefit from the ability of UV to help generate “press” credentials, for everything from music awards to the Olympics,” he said. […]
[…] not news, I was on about this last summer as well: Actually, it might not be the bloggers they’re getting over on, it’s more […]