Wednesday, May 7th, 2008
Accountability: The Evolution of the eBay Feedback System
Yesterday I sat down with Brian Burke, Director of Global Feedback Policy, to get a full run-down of all the changes being applied to eBay’s Feedback system. During our discussion, I was able to get the rationale and reasoning behind the changes as well as ask some of the questions that had been provided by Ink readers over the past few days.
Before I jump into the overview, interview and presentation, I wanted to give my two cents. The overarching theme coming out of these changes seems to be a trade of transparency for accountability (hence the title of my post). I’ve received a few emails stating that this is very “un-eBay”. However, I get the impression that eBay’s public feedback system was always intended to provide transparency and accountability but that over time, the transparency of the feedback system has taken us away from the accountability goal. So, with these changes, eBay should return to a more healthy balance of the two.
NOTE: I’ve just checked out the AB post about the changes and in retrospect, I am definitely toeing the company line on this one.
Here is a breakdown of what is being introduced this month:
1. In order to encourage repeat transactions and reward good service, eBay will provide credit for similar repeat transactions going back to when the system was introduced in 1996.
2. In May, sellers will no longer be able to leave negative or neutral Feedback for buyers.
3. eBay will remove negative and neutral Feedback left by members who are suspended or who fail to respond to the Unpaid Item (UPI) Process.
4. Positive Feedback percentage will be based on the past 12 months of Feedback, rather than lifetime on the site.
5. Restrictions on when Feedback can be left:
- Buyers must wait seven days before leaving negative or neutral Feedback for active PowerSellers who have been registered for at least 12 months.
- Members must leave Feedback within 60 days of the transaction closing (today members have up to 90 days).
6. eBay is removing Mutual Feedback Withdrawal.
This is a global eBay site-wide initiative which will start to roll-out in May, 2008.
5/12 Australia
5/15 UK, Ireland
5/19 US, Canada, Canada.FR
5/20 France, Spain, Italy, Netherlands, Poland, Belgium.FR, Belgium.NL
5/21 HongKong, Singapore, India, Malaysia, Phillipines
5/22 Germany, Austria, Switzerland
Additionally, he provided me with the presentation that summarizes the evolution of the feedback system:
Q&A with Brian
Can you please explain how the 2004 and 2007 ratio of retaliatory feedback numbers (shown on slide 5) was determined?
We used a really simple definition when determining exactly what constituted retaliatory negative feedback. It was strictly a user who received a negative feedback and subsequently left a negative feedback. That second negative feedback was counted as being retaliatory. There wasn’t anything that we did to go back and look to see if that negative was justified rather we observed how behavior changed in the marketplace over time. We saw that 4 years ago sellers would do that two times as more frequently than a buyer and today it is eight times more likely.
It’s basically how our members interpret retaliatory negative feedback so we wanted to keep it simple; getting a negative after receiving a negative constitutes retaliatory feedback.
A week or two ago the eBay developer blog announced “We will base the Positive Feedback Percentage on the past 12 months of activity (and will include neutral feedback in the calculation.” Is the above quote still accurate? If so, why are neutrals now going to count as negatives?
The above quote is accurate. As for neutrals, they’re not counted as negatives but they’re going to not count as positives either. When we first introduced the feedback percentage we had, within our feedback system, something that was called NARU neutrals. Basically, what had happened up through 1999, whenever a member got suspended we took any feedback that they had left (whether it was negative, positive or neutral) and we converted it to Neutral. The majority of feedback that was left for users was predominantly positive so we didn’t think it was fair back then to include NARU neutrals in the overall feedback percentage so we chose to count only positives and negatives in calculating the feedback percentage (although ideally we would have liked to track all). Now we’re going to a 12-month window, we no longer have to worry about NARU neutrals unfairly affecting an overall feedback score. So what we can do is make the Positive Feedback Percentage be exactly that. What’s the percentage of positives that a seller has received over all of the feedback they have received? Which is really what sellers should be measured against.
How is eBay going to protect the sellers? How is eBay protecting the sellers from bad buyers? How can I truly block an eBay bidder given that anyone can create a new disposable eBay ID at any time to circumvent my blocked bidder list?
First, what is good for the buyers is ultimately going to be good for the sellers in our Marketplace and we’ve already seen a lot of sellers changing behavior and focusing on what’s most important to the buyers just by the announced changes we’ve made.
When we made this change we understood clearly that we were taking away both a real and a perceived protection. For the seller who had never left a negative – it was just a perceived protection that they had. We needed to make sure there was a balance to the system. The goal behind the changes was not to enable buyers to just leave negatives. Rather it was to go back to the original intent of the feedback system - to make sure that both buyers AND sellers are held accountable for their actions in the marketplace. We’re going to hold sellers accountable through a public reputation system, leveraging our buyers. The way we’re going to hold buyers accountable is through private reporting from sellers; through enhancing the tools that we’ve given sellers to help protect them from buyers. We’re going to rely on sellers to let us know when a buyer has violated policy.
Why is my blocked bidder list no longer alphabetized and why are some seller’s blocked bidder lists disappearing all together? How can I truly block an eBay bidder given that anyone can create a new disposable eBay ID?
We haven’t made any changes to the blocked bidder list system so I’m not sure why someone would be experiencing that. I’ll have to look into that further. I haven’t heard about blocked bidder lists disappearing all together so if you could get me an example I will get that to the product team to find out why that is happening. In the case of a bidder circumventing a seller’s blocked bidder list with a new ID, that is against the rules and there has been no change in policy here at all. If that buyer is identified, he/she is suspended from the site.
I rechecked Feedback criteria today and it seems that the reason eBay couldn’t censor feedback was due to potential legal liability to eBay if they did. How do the shifts in feedback procedures pan out legally? Does the shift in policy constitute censorship?
In the United States only, eBay is given protections as a hosting service for the comments and content that another person puts on our site. As we don’t actually edit the comments or make changes to the comments – there is no difference in how we’re protected. All we’ve changed is how a user is rated in the marketplace. We’ve always had rules in place that allow us to remove negative feedback or positive feedback and the comments – but we won’t go in and edit the comments. Under the US law we can’t do that. To be clear though, we don’t get those same protections outside of the US but we still have a very similar set of rules so even if that law was to go away in the US we probably wouldn’t make changes to the system because we’re trying to design a system that instills trust within the marketplace, not one that is in place because of a US law.
What steps did eBay take to educate the users, both buyers and sellers, during the 3-year period that showed a trend in retaliatory feedback?
Well, the focus on retaliatory negatives didn’t just happen overnight. We examined a number of other alternatives before coming to the conclusion that the feedback system needed to change. The one thing we learned was that when a buyer gets a negative feedback in the eBay marketplace, they discontinue participating in the marketplace and it’s not healthy for anyone. All of the alternatives we examined still didn’t address one of the leading reasons why buyers were not comfortable continuing to participate in our marketplace.
There’s been a lot of focus on buyers but what are the changes we’re making to protect sellers?
One is basing the feedback system on a 12-month window rather than a lifetime window. For example, in the past, a seller’s feedback score has been negatively impacted by a negative left 7 years ago even though it doesn’t necessarily accurately reflect the seller’s participation in the marketplace today. So they’ll no longer be dragged down by old feedback and it’s more relative information for the buyer.
Second, if the buyer receives an Unpaid Item Strike and fails to respond (and that occurs more than 70% of the time today) we’re now going to remove the rating and the comment (in the past we just removed the rating). In the future, if a buyer responds in a manner that doesn’t fault the seller, we’re going to take a look at what buyers are saying in the UPI console and remove it if it is arbitrary.
We’re going to remove all negative and neutral feedback when a member gets suspended.
We’re going to prevent negatives and neutrals being left for PowerSellers that have been on the site for more than 12 months (within 7 days of the transaction end). This is a big deal for us because we’ve always felt that we shouldn’t be able to prevent a buyer from leaving a negative feedback but we feel that PowerSellers who have been on the site for 12-months are pretty safe. We’re going to start with that group of sellers so that if a buyer tries to leave a negative or neutral within 7 days of the transaction end, we’re going to provide a link that encourages communication between the buyer and the seller to see if the buyer has given the seller enough time to deliver on the transaction.
In order to cut down on potential extortion situations we’re reducing the number of days someone can leave feedback from 90 to 60 days.
For Cross Border Trade we’re going to include messaging in the leave feedback flow to remind the buyer that it was a CBT transaction.
When a buyer goes to leave a negative or a neutral – and very single buyer will see this, every single time they leave a negative or neutral – we’re going to ask them three key questions:
1. Have you communicated with the seller?
2. Have you allowed enough time?
3. Have you kept the feedback factual?
The big difference between then and now is that before, buyers saw the above questions but moving forward, they will have to check each question before proceeding. So it is creating a speed bump for buyers in leaving a negative or neutral and we hope that it will educate buyers, the first time they go to leave a negative or a neutral, that it is important to communicate, it is important to allow time for shipping, and that it is important to be factual.
When satisfied buyers attempt to leave DSR ratings of all 5’s they are prompted with the following message: “Are you sure? Remember, the ratings left by you are anonymous and can’t be seen by the seller.”
Why does eBay feel they need to doubt the satisfied buyer’s desire to leave 5-star DSR ratings?
The short answer is we don’t do it just for those with 5. We do it for anyone who is leaving detailed seller ratings. Whether someone intends to leave a 1 or intends to leave a 5, they’re going to get that message. This isn’t about us doubting someone whose desire is to leave a 5. Rather, it was an effort by us last Summer to address an issue we saw arise when we first introduced detailed seller ratings. Our conventional feedback system is a public system that is transparent to the marketplace and we wanted to communicate to the buyers that DSRs were designed to be more anonymous with the ultimate goal to obtain more accurate, honest feedback from buyers.
Since eBay has made such a point that DSRs are anonymous, what faith do we as sellers have that eBay is accurately reporting the ratings that our buyers are actually leaving for us? While still protecting individual’s anonymity, why not provide to me some DSR information of value? For example, what is my DSR rating from international customers versus domestic buyers for each DSR?
To answer the first part of the question… it really is in eBay’s best interest to accurately represent how buyers are reporting on sellers because that will favor those sellers that are performing best which is exactly what we want.
With regard to seller reports and information, I absolutely believe this is an area in which we can dramatically improve. We’re introducing the Seller Dashboard that provides basic information to the seller about their DSRs but we’ve heard from a lot of sellers that they would like to slice and dice that information so we’re working on a separate project right now that will enable them to either create their own reports or for us to provide a much more robust reporting structure that our sellers can take advantage of. All I can say on that is to stay tuned; it’s not something folks are going to see in the next couple of months but it is something we’re working on.
Brian, thanks very much for taking the time to provide answers to Ink readers. It is greatly appreciated and we look forward to having you back on the blog soon.
My pleasure. Anytime.
Tagged: brian+burke, buyers, detailed+seller+rationgs, dsr, ebay, ebay.com, ecommerce, feedback, Marketplace, online+marketplace, sellers
Scott @ TradingAssistantJournalOn 05.12.2008 at 6:47 am Said:
@IMPLOG
Yes, I agree, your question should be answered. Many questions should be answered. I never pointed out anyone in particular as being combative so do not think my comment was directed specifically at you or anyone else.
Keep asking the question, it is a great question and I agree with you completely. Your point is valid, and there is no reason why this flaw could not or should not be corrected. There is no reason that I can think of from the outside that justifies why your question has not been answered or why this problem has not been addressed.
As I said, we are contributing to the conversation here. You are contributing and so are many others, and this is why eBay created this new blog. They did not start this, and invest in it, hire Richard etc… To just simply do business as usual. (At least that is my most fervent hope).
Your consistent message has been heard, why it has not been responded to by now, is a mystery to me, but I am sure there is an internal reason. Whether that reason is valid or not, we will not know, unless Richard can get your answer. Which I believe he is working on.
Let us hope it is a positive and worthwhile answer when it comes.
JJHOn 05.12.2008 at 7:32 am Said:
Not all comments on this blog have done this but a good portion of the comments here have been less than fair to the messenger.
Funny, I don’t think I’ve read ANY comments that have directly targeted Richard. They are simply comments from angry frustrated sellers, that are hoping to use this medium to be heard.
Are they heard? You say they are. My take is “they are, but the issues are ignored, and will CONTINUE TO BE IGNORED until they are seen in profits being lost”.
The response is Ms Norrinton comes on the blog and leaves a “we love you” message that answers NO issues or previously asked questions.
…and you think THAT doesn’t throw fuel on the fire? I think that’s worse.
I am still waiting myself for the answer to TWO directly asked questions. Question that would take a moment. All Richard has to do is send an email. Maybe he has - maybe the exec he sent it to isn’t replying. Meanwhile I’m left with no answers, no clarification, just silence.
So again… Why am I, a 10 year veteran, once a Powerseller, 100% positive feedback, and 4.8 averaged DSR score NOT CONSIDERED TO HAVE A [good] TRACK RECORD just because I’m not a Powerseller?
[Sentence edited: see comment policy]
You want to talk about fuel on a fire. Some body else is using a fire hose to apply it, and it isn’t us. By the time eBay realizes the damage that’s been done, it will be too late.
Patricia1On 05.12.2008 at 8:44 am Said:
Scott - it seems to me when ebay opened up this blog for public comment they should have assumed they would get more of the same. Every article I’ve read that contains comments and every blog I’ve read that contains comments are just like the comments here. Hello? Sellers ARE angry! Just because some are vocal and some are not don’t think for one minute that the angry ones are only a small bunch of big mouths. The quiet ones are leaving or will leave the minute they personally get burned by the new policies. That’s a given…a fact that cannot be denied. I know sellers who never post on boards like this and - funny thing - their thinking and their feelings are the same as mine and the same as every seller who is posting here! If ebay truly wants to keep their small sellers they had best listen - go out and poll small sellers - find out what they really are thinking because they’re ARE losing them. If that was their intent in all this (and I kind of think it is) then I’d say they’re right on track.
JJHOn 05.12.2008 at 8:48 am Said:
For the life of me, I can’t imagine what got edited? I didn’t say anything wrong.
AmberOn 05.12.2008 at 9:02 am Said:
“They did not start this, and invest in it, hire Richard etc… To just simply do business as usual.”
Unfortunately, that is what they are doing. Sure, they respond now, but their responses are as empty of actual information as they’ve always been (Usher’s posts excluded IMO)–and even MORE dismissive of our thoughts, concerns, and business needs.
I think Richard is doing a terrific job of attempting to get some answers–but I also think that he is being stonewalled with the rest of us. 
It is impossible to understand the depth of seller anger without experiencing the causes. It’s a festering wound. It is not one particular policy, but the gradual shift in policies and attitudes built over time that are causing the seller anger and exodus.
The very fact that so many posts here ARE so full of obvious anger and so combatitive in tone should tell you something…and it isn’t good for eBay. Sure, sellers have always whined and complained when changes were made. But I can’t recall in the last 5 years when so many long-term professional sellers were so unified in their disgust of the company. Not even the SIS, Store Fee Hike fiasco can approach the real, palpable anger many sellers feel right now.
Ebay shouldn’t, in all honesty, worry about posters like me–after all, they have buy.com to take my place. What they should worry about are the hundreds of sellers just like me who are leaving without a word. Those who close up shop and move elsewhere without ever posting to a blog, discussion board or sending an email to ebay. That quiet, but equally disenchanted demographic is the one that is going to cause eBay problems because once lost, you aren’t getting them back. Ebay’s traffic advantage is gone, it’s more expensive to sell here, and it has a poor customer service reputation. Until all 3 circumstances change, I think we’ll see a steady influx of large retailers on the site to make up for the mass exodus of smaller sellers like me.
SandiOn 05.12.2008 at 1:24 pm Said:
When an executive of Lorrie Norrington’s stature takes the time to respond and to clarify statements in a forum such as this, I think everyone should notice some changes have occurred on the other side of the fence.
But she failed to clarify anything, that is the issue, she merely gave us more lip service with no substance.
And there also appears to be some people who do not even use the eBay site at all who think it is quite fun to add fuel to an already hot fire.
I am so tired of this statement. As someone who does not buy or sell on ebay presently, that does not mean I have lost my right to voice my suggestions, concerns. I still respond to questions at the answer center, I am still counted as 3 users by ebay on their stock reports and reports to the board.
Perhaps those of who left would like to be able to return, but not until the environment itself is healed - and is moral and ethical to ALL members.
I would say the same thing as you did in reverse - just because ebay doesn’t like the questions, does not mean they are not valid questions.
We are now reading original content from within the ranks of eBay on this blog which can not be found anywhere else.
That’s inaccurate, to date everything can be read elsewhere. I’ll admit I have never read your own blog scott until last night, you have more original content from ebay management.
ebay management should take lessons from Colin Rule, he has consistently been “out there”. When he posts anywhere you always know he means what he says, he will answer the hard questions. I may not always agree with him, but he always gives a complete overview why a decision has been made.
I will believe ebay is listening when they address the user verification. That single issue has been brought up for years by users. That one issue is responsible for more bad buying experiences and bad selling experiences than any other single thing.
Instead of correcting it, instead of addressing it - ebay managment team has simply compounded it with the new feedback policy. Every seller would simply like to know that in the event a buyer can really be NARD that they can’t simply sign up again with a new user name.
It’s so basic, would take less thought and energy than anything they have done to date, and would solve more problems than any of their other so called fixes to date.
It is so fundmentally simple, so fundmentally logical.
Yet it remains a thorn, why?
As long as that single issue remains, ebay will be viewed as simply giving lip service to the buyer experience, making the site safe to all.
The funny thing about that is buy can’t afford to be taken by fradulent buyers, they almost lost their merchant account because of that a couple of years ago.
SandiOn 05.12.2008 at 4:15 pm Said:
My ebay email brought up a new question that I emailed ebay and asked about last year with no response.
Why does ebay seem to time changes in feedback to coincide with USPS rate increases? It really isn’t very seller friendly and might in fact further instill buyer’s misconception of what postage really costs?
Granted I personally believe ebay’s push for free shipping is based on making more money for themselves, not the buyer experience, but I think this is one of those actions that actually have an effect ebay didn’t think out fully.
They should be on top of educating buyers on reality, and making changes that can so easily be impacted by 3rd parties – USPS and their rate increase – fail ebay and the sellers alike.
Might even be nice if ebay acknowledged the fuel prices buyers are paying at the pump are also impacting what sellers are having to pay to ship items.
It would at least give the impression ebay gave a rip about sellers, large and small.
PS: Scott and Richard, since you both have brought up posters who longer buy and sell on ebay, Richard, you specifically pointed to me regarding this in one of your posts. I bought a share of ebay stock today, so I may no longer buy or sell on ebay, I am a registered user x 3, and a stockholder. I have a right to my voice.
MistyOn 05.12.2008 at 4:45 pm Said:
“For the life of me, I can’t imagine what got edited? I didn’t say anything wrong.”
JJH What is interesting is that they can take the time to edit but not respond.
As for this small seller I am done talking. When I start to see changes for the better being made then I may reconsider, I think it would be wise for others to do the same.
Yes, actions speak louder than words.
implogOn 05.12.2008 at 5:11 pm Said:
The new Feedback “enhancement” is now in effect in Australia. Look what pops up when an Australian buyer attempts to leave positive feedback for a seller.
“Important changes to Feedback
Buyers, you can no longer receive negative or neutral Feedback from sellers.
You should leave honest and accurate Feedback without the fear of receiving negative or neutral ratings”
http://forums.ebay.com/db2/thread.jspa?threadID=2000566659&tstart=0&mod=1210630357309
JJHOn 05.12.2008 at 5:44 pm Said:
I’ve been thinking hard what got cut. I think it was a comment about ebay would not notice that they made mistakes with all the changes until their profits start to drop. If that is what got cut, then I can’t imagine why, unless I touched a nerve. Otherwise that’s just plain censorship. But then “eBay may, in our sole discretion, [may] reject comments for any reason we deem appropriate.” Whatever.
BrendaOn 05.12.2008 at 6:12 pm Said:
Ebay, I’m flabbergasted. I really have to begin to believe that the kool-aid rumors are true.
I read policy and following statements and I’m really beginning to believe that the top brass may be mentally unhinged.
Allow me to clarify, before I manage to get the company at its sole discretion, regect my comments because they might make sense.
How can the company say that it is bhind the small seller when in order to derive any of the potential protective services that Ebay has at its disposal a seller has to be at power seller status?
EXAMPLES:
1. Being able to speak to a customer service rep
2. Earning a 7 day wait time before feedback from a buyer ( Isn’t bad performance regardless of number of sales generated? )
Shouldn’t this said seller with a proven track record be better able to satisfy a customer and not be in need of additional coddling?
When I researched Bain & Company, they seemed to value education and training. Did 3 years erase that standard from Mr. Donahoe?
I am curious. How does Ebay say they value small sellers and then by policy dictates force them to function at a higher standard than the pampered power sellers? Well, getting to a solid 98% with no real safety nets is a phenominal feat.
Another question that prompted me back. I visited the Australian AC and found a link to the proposed feedback policy changes. How did the Aussie’s manage to merit well written sentences that clearly explained what was going on? Don’t American users merit decent sentence structure? ( this was a 1% possibility moment that has kick started my internal radar. )
Well, you see, it stood out, and bears closer examination. The posters in the Aussie Ac didn’t seem to be rabidly upset about it. They’re up in arms over Paypal, but this, eh.
Sadly, the more I poke around, the less things seem to jive out into any semblance of a workable business plan.
The site is decidedly favoring power sellers. Actions speak louder than words - big time.
But yet again I will extend a HUGE thank-you to Ebay. Your very poorly run company showed me the neccessity of being extremely cautious when accessing a business venture. Caution is not always my strong suit. Your lack of transparency and complete lack of respect of your users will continue to serve as excellent reminders to me as I begin my e-commerce career elsewhere.
And while growing up, my father’s standards were at 100%. I am able to produce this. It’s a pity that Ebay can’t.
Patricia1On 05.12.2008 at 6:44 pm Said:
“JJH What is interesting is that they can take the time to edit but not respond.”
Don’t understand…since they must have all the time in the world. I sure don’t see anybody else rushing here to leave an opinion. In fact, I read a post about this blog on Seller Central. The poster who asked was told - “yeah, ebay has a blog called the ebayinkblog dont’ wast your time”. Its over there in black and white. Seems everyone knew this blog was set up to bring ebay-speak to the masses and the masses aren’t having any…maybe we should take heed of that and stop senselessly spinning our wheels 
DawnOn 05.12.2008 at 7:29 pm Said:
On ebay.com.au discussion boards, it’s being reported that buyers receive this message when trying to leave positive feedback for a seller:
Important changes to Feedback
Buyers, you can no longer receive negative or neutral Feedback from sellers. You should leave honest and accurate Feedback without the fear of receiving negative or neutral ratings
Richard, can you confirm this?
SandiOn 05.12.2008 at 8:00 pm Said:
On ebay.com.au discussion boards, it’s being reported that buyers receive this message when trying to leave positive feedback for a seller:
Important changes to Feedback
Buyers, you can no longer receive negative or neutral Feedback from sellers. You should leave honest and accurate Feedback without the fear of receiving negative or neutral ratings
Richard, can you confirm this?
If it’s true, why not simply say:
“Please leave a negative, it’s on the house.”
Patricia1On 05.12.2008 at 8:32 pm Said:
LOL how stupid is that? Its actually more sick than stupid! Apparently ebay is no longer in the business of letting sellers makes sales so they can collect their fees. I have no idea what business they’re in now…
We close the comments for posts after 30 days. If you would still like to comment on this post, please use our contact form.
177 Responses on this post. Click to add yours.