Hi all – the results of our homepage a/b test are in and they aren’t super conclusive.
The new homepage is performing slightly worse than the old homepage from a conversion standpoint. Conversion, a measure of determining how many visitors purchase, is the most important metric in this test. Bounce rates, those who arrive on the page and leave immediately, were the same for each version. The results are so close that they probably aren’t statistically relevant, but it seems clear that the new version is not a significant improvement over the old one.
Obviously we were hoping for better results, but this is precisely why we run our a/b tests! We’re going to revert 100% of our users back to the old version while we plan what’s next.
What’s next? Another test where we try a different approach but consider the data of the previous test. Look for that in the next two weeks or so.
- Alex
28 responses to Homepage a/b test results
Thanks for the update. Glad y’all test things out to figure out what works best.
How about you change the test…keep the new one on all weekend long and compare it to last weekends ? Keep it up for everyone to see, at the same time, and maybe next weekend change it to the old one again and see what happens.
@shellbysbooth – We ran both versions in parallel for a full week. As tests go this is as clean as it gets; running each version at the same time. If we start testing different time periods we run the risk of adding outside dynamics into the data (traffic fluctuations, etc)
That said we are going to run another test soon!
Maybe it’s me. Most people come in directly thru google in searh of something specific. Lack of sales can be 2 things…lack of name recognition….ask your friends and neighbors if they’ve ever heard of Bonanzle. The other is the shippings costs..most are over-inflated due to the calculators. If you do free shipping as a means of a work around the prices look even more ridiculous… the biggest turn off of all
TY for the update! Love this site and what it has to offer, just wish I had some sales
Alex, why not use the new version as the SECOND point of contact when buyers enter the site? It is very crisp and clean and great to see that categories list down the left side. It seems like it’s a shame to toss all that work. I have mentioned several times, the new version looks like the Table of Contents page of a book. The old version looks like the book’s cover. Both are important to the book, but that old gorgeous home page is like a pretty gilt cover that makes the reader grab the book!
@STB – we’re not throwing away the work. We’re going to try again with a revised version that leverages the work and data from our last attempt. Based on your feedback I think you’ll like the new direction we’re considering. ; )
well Boyz isnt there a saying that bigger is better and when you have full page HPLs its better or in this case just as good as the test ….looking forward to seeing whats next !
What I love about you guys (the Bonanzle team) is 1) you communicate 2) you actually DO test! 3) you are not afraid to back up and try something else… btw, did I say I love you guys?!
Thanks for the update, Alex. It’s nice to know that if a new idea isn’t performing, you’re not afraid to back off and try something different.
With that in mind, I’d sure like my booth defaulted to Newest First, and I can’t afford membership yet!

To me the test says people (what few there are) come in thru google, and bypass the home page. They don’t come here to look or browse. They come to buy the item they were searching for on google, and then they leave. THAT is the problem; not the home page. They have to know the name Bonanzle, before they’ll actually come here…
I agree with Sofy. The back end needs to work intuitively, perfectly… because that’s where the sale is lost or where the money is made. The rest is frosting on the cake.
@alpenquin Most of our shoppers do indeed come through Google and bypass the homepage as you state. And agreed we need to build the brand and name recognition. Fortunately we are looking at a number of different customer acquisition methods to do just that.
Even if someone knows the Bonanzle name, they might just not come in through the home page. My bookmark does not go to the home page, but My Bonanzle Page. If there is a specific search the Buyer always does, they might just Bookmarked that search.
I agree with what Sofy said above.
The old homepage was/is very nice, but
the main effort as I see it is to garner name recognition for the site so buyers come here to search and shop.
And once the buyers are here, an easy checkout, an easy best offer button and an easy calculator with shipping options will lead to repeat happy buyers wanting to come back again.
How about putting wires one the face and head of testers to see where there eyeballs go, and see what they click when they see each test version?
Ok, just being silly.
Keep up the good work at attempts to improve a sight that already looks pretty awesome from initial impressions. Navigating is another issue, from buying and selling viewpoints.
Hi, Alex ~
There’s another site that starts with a B that has a home page that’s so spectacularly beautiful & interesting that a lot of people (including me) go there every day just to see the new one.
It seems to me that it’s a worthwhile goal for Bonanzle to produce a home page that’s just as irresistible as that one. I believe this could be done with the current HPL showcases if the best of them were selected on a regular schedule & refreshed automatically instead of the current aggravating hit-or-miss setup.
I got interested in Bonanzle when the HPL showcase was introduced & returned many times specifically to see what the current one was & then stayed to browse & buy.
I think the showcase concept could easily be built on & developed into something quite unique to give Bonanzle a very distinctive signature & set it apart from other sites, making it a destination site.
Bonanzle is fortunate enough to have some very talented & dedicated volunteer list makers whose best efforts approach or meet the standards of professional quality work. Their best work truly can be described as list art. It just seems to make good sense to build on this asset that is already in place, well received, & best of all ~ free!
To me, making the home page a distinct & conscious destination is every bit as important as focusing on the technicalities of making the site more easily navigable. What good would it do to accomplish that aim if few people visit the site through that page anyway? Getting them there is the first place should be goal #1, in my view.
I like it that you’re so willing to go back to the drawing board to reconsider the options for making the home page a place that really reflects & enhances the Bonanzle that its community knows & loves. I know you will give it your best to help move it forward along the road to even bigger & better things.
Your hard work is much appreciated!
I think I agree with STBThreadworks and encourage you to proceed to plan C.
A good HPL is spectacular, I have bought an awful lot from them. There are some unattractive ones too. The ‘B’ page was a good clean design and had it’s own merits but IMO it detracted from the HPL. That said I know that while I get views from within Bonanzle, the majority of my sales come in off the Google expressway, pay and are gone. No muss, no fuss ka-ching! I do not have a problem with that either. Any sale is a good sale.
Those who are playing the ‘advertise at all costs’ song probably will not want to hear this. The old days of going to hang out on eBay and see what you can find to buy are over. They are over on eBay. Why would they be reborn here? A business plan that relies on that concept is going to fail.
In the same way that eBay buyers grew up and realized that the ‘rare once in a lifetime unique gotta bid now’ item was generally ‘none of the above’ and another one would come along, internet shoppers have generally matured.
As a collector I check on Bonanzle first because it is my place, if I do not find it here I look on Google. If it is priced way too high I hit the back button on Bonanzle too, because I am not stupid. I do not believe I am unique.
Sales are poor almost everywhere, ordinary people are afraid of the economy and they are conserving their money, if they have any. I would encourage all sellers to search for comparable items to what they are selling, both here and on Google and revise their pricing accordingly. If another seller is engaged in suicidal undercutting hold your price but don’t be surprised if it sits until he or she goes out of business, which they will, eventually.
Thank you for treating Bonanzle sellers as rational beings who are vitally interested in what happens on the site, not like mushrooms.
Wow, those are some extremely thoughtful comments, all much appreciated. I am racking my brain and I can not think of a site that starts with a “B” that has a home page that’s that good. Maybe you can email me the answer Northwest?
I feel… in my humble opinion … the new page was just too busy. and really too ‘generic’ .. like everyone elses.
The original page, although plain to some has a unique favor that causes people to stop and look.
I agree with Felicitations – the majority of my sales come from Google as well and Advertising, although it DOES play a vital role, it is not the be all end all to driving traffic here and Branding.
The Bonanzle brand is gaining ground, and only through good quaility items, at good prices and excellence in customer service from all sellers here will the name Bonanzle stick in the consumers mind.
I also agree making sure the shipping, shopping cart , etc is vital to good customer experience. If they have any trouble, they will not be back.
As far as Branding goes, I have seen my own "brand’ evolve from Renagades Relics to Renagades of Music, I did not bring about this transition Deliberately, it evolved and I just adjusted accordingly.
Bill has a vision … Bonanzle is his heart … by reflecting there more and less in what works for other marketplaces you will see the direction Bonanzle needs in order to lock in and evolve the brand.
Ditto what Felicitations said.. couldn’t possibly say it any better myself.. and I’m wracking my pea brain trying to figure out the other “B” site too.
I think I know the B site you are referring to. Great minds….
Bing-o, Alex!
Ok – I just saw it for the first time. Spectacular! and I even clicked on a few of the squares. Never felt compelled to look at the site before…
Yes that page will keep me coming back!
Wonder if there is a way to have the classic glam Home Page have a little square showing the item that was clicked on via Google – like the “B” site has little squares – only the items gallery photo would actually show super-imposed and be clickable to the item seller’s page… ?
Does that make sense?
well as long as our individual stats show that 90% of my sales (and most of the other primie members i’ve spoken with agree) come from google and bypass the home page; it seem like we should be trying to now figure out how to get people to the site other than from google. Which brings us all the way back around to point A. Point A = advertise. So much time ( which is $) has been wasted on every other method other than 5 second spots on cable TV to get bonanzle’s name to be recognized yeah yeah yeah, it’s expensive; but so hasn’t all the revenue lost (not made) over the last two years and all the changes that amounted to goose eggs. Personally I like the new homepage and think the only thing that needed to go was the questions and answers ditty. Buyers don’t care about sellers questions.
I’m also at a complete loss as to what the other “B” site might be.
The one thing about here that has always kind of not set right with me about here is what I perceive as a “bargain” mindset. I feel that the public could grow accustomed to expecting sales, and knowing that sales WILL come along, will learn to wait for them.
Are there any ways to shift the emphasis a bit towards the uniqueness of this sites offerings and that there is the ability to chat directly with the sellers, which creates the opportunity for the BEST in customer service? I keep reading about the shifting over to social commerce and it seems to me that Bonanzle has all the necessary tools to accomadate this approach pre-built in.
I would also like to see more actual product pictures in the monthly emailings. “Don’t make me look”, show me, and tempt me that way. Would it be possible to incorporate the HPL’s into the mailings?
It might also help to have an official Bonanzle presence within Twitter. Twitter blips make it into Google search land.
renderBonanzleWidget();Collector & Buyer
Login to see more comments