Why doesn’t Trade Me have an API?

This is a question I get a lot:

Why doesn’t Trade Me have an API?

It’s actually a slightly frustrating question for me to answer.

Internally I’m usually the one asking this question. Externally, at places where technical people gather, I’m the one defending the fact that we don’t have an API and, what’s more, have no immediate plans to build one.

Why not?

Nat Torkington’s recent post has some of the answers.

It’s not that we haven’t thought about it. There are some legitimate reasons why we’ve chosen to not build an API to date. I thought it would be interesting to talk about some of these and get your thoughts.

Some questions to think about

Would we need to communicate all changes in advance to third party developers? If so, how much in advance? We’re constantly making small changes to the site. We generally deploy site changes twice per day. The cycles can be very short. We sometimes deploy something in the morning and then tweak it later that afternoon. Anything which threatens to slow us down is quite correctly frowned upon.

What happens when we need to make breaking changes to the API? Do we version the API and continue to support older versions? If so, how long do we leave this support in place? If not, what liability do we have if we break a third-party application?

How do we deal with authentication? We put a big effort into keeping Trade Me safe for buyers and sellers. We have a full-time team working on this. One of the problems this team deals with is phishing of members login details. We have a simple and consistent message for members: don’t enter your Trade Me email address and password anywhere other than on the Trade Me site. So, obviously allowing third-party developers to build tools which require our users to enter their login details is inconsistent with this. To solve this we’d need to build an alternative authentication process – e.g. the token based approach used by upcoming.org.

Are we prepared to invest in creating an eco-system where third-party developers can profit? As Nat pointed out to me when I discussed this with him, one of the reasons that Amazon have been so successful with their new web services is that they are creating more value then they are capturing. In other words, they are leaving some money on the table for the people using their API.

Are we prepared to allow our customers to become dependant on a third-party tool? If somebody created a really wicked tool using the API, and lots of our users started to use it, would that limit our ability to innovate that the same area? This is a dilemma that eBay have started to encounter with their API, where they have created listing tools which compete directly with third-party tools built on top of their API. At the moment I’m not sure we’re prepared to let others build something we then wish we had built. Is that bad?

How do we protect the user experience? How do we protect our brand? We’re currently very protective of both of these things, for very good reasons.

How do we protect our infrastructure? In the past we’ve had to ask people to discontinue or specifically block access to automated external tools which were causing us pain. To an infrastructure guy there is a fine line between a well-meaning but poorly implemented external tool and a Denial of Service attack. In fact, we currently prohibit the use of any “robot, spider, scraper or other automated means to access the Website or information featured on it for any purpose” in our Terms & Conditions (see 4.1 c).

If we build it will they come? Are there enough developers in New Zealand to justify our effort in creating an API? How many people will actually use it? How many people will use the applications they build on top of it?

Do we have bigger fish to fry? Keep in mind that any development work required at our end would be at the expense of something else. Is an API just too much work for us for too little reward? Any argument in favour of an API need to be more compelling than: all the cool kids have one. :-)

Your thoughts?

What would you do if you were in our position?

19 thoughts on “Why doesn’t Trade Me have an API?”

  1. Hi, Rowan. We talked about this at Kiwi Foo Camp, I seem to remember. I don’t think you would have convinced me with those arguments, though.

    While theyt sound like good arguments, they also are very weak in the face of the fact that eBay has APIs and makes a shitload of money from them. 40% of their listings come from their APIs rather than from people typing stuff into web pages. Your reasons why you don’t have an API sound more like manufactured excuses, obstacles you’ve found to justify your inaction.

    eBay solved authentication, they solved updates, they solved ecosystem, they dealt with user experience, they sorted out the infrastructure. These are solvable problems.

    Trade Me’s rate of development isn’t exactly blistering. I’ve seen the dev team–you don’t have a lot of people. I think it’s a bit rich to say that you don’t want other people to build things you might eventually build yourselves. I’d be more inclined to accept that argument if you were likely to get to new features. And, don’t forget, while you sit worrying about what you *might* do at *some* point in the future, your users don’t have the features. It’s not a particularly user-first mindset you’ve got there.

    A cynic might say that the real reason you don’t have an API is because you already own the sector. Why spend money to develop something? So people will find it easier to list more items? They have nowhere else credible to go. You have a shitload of business. There’s no pressure to wring more out of the customers you already have. The hell with it, make ’em use the damn HTML form and they can be happy about it!

    I feel I’d learn a lot more about why you’re not offering an API by asking: why would something that worked for eBay not be something you’d want to do?

  2. I’m not a developer so I don’t have a strong opinion on the API – although I think that anything that helps me utilise the full potential of TradeMe is a good thing.

    But I do have a strong opinion on the lack of RSS integration into the site. I’d like to see RSS feeds for everything: member listings, category listings, saved searches, $1 reserves, closing soon, jobs, property, etc…

    I think that lots of NZ websites are afraid to offer feeds as they believe that this will stop people from visiting the main site. Those that do offer feeds, don’t provide full-text feeds, for that same reason. The idea is that if you offer a partial text feed it will encourage users to click through and visit the main site, but this has been proven to be untrue. (See FeedBurner blog for more on this.)

    I’ve created a custom RSS feed for the Wellington Jobs section, but I didn’t realise that this is probably breaking your TOS as you mentioned above. If you offered your own feeds this wouldn’t be necessary.

  3. Hi Rowan,

    For reasons why you should build an API — to make the lives of your users easier!

    With such a wide user base, TradeMe is probably never going to be able to cater to each segment of the market — perhaps DVD buyers need certain tools, but how about rare book buyers, or silverware collectors or baby clothing buyers?

    There’s the obvious use case of the serial seller, who currently can use a product like Auctionitis to give them extra management features. I understand that Auctionitis just scripts web page access, and so is probably a lot more fragile than it would be if it could use an API.

    Also the buyer — currently it is quite hard to keep track of the auctions that you are following, and how your bids are faring, whether you have been out-bid and when the auction closes. There is email and sms support for these, but a desktop app (perhaps similar to the TradeMe gadget) would go a long way to making this experience better.

    There are definitely enough developers in New Zealand to build on top of an API, as is witnessed by the number of little TradeMe hacks that pop up over time. I think that there’s always an eco-system around major websites, and there’s a real opportunity for support systems for TradeMe sellers and buyers that can make their experience much richer.

    Authentication is a very important issue, and for most users I agree that it has to be handled carefully as they may not understand the ramifications of using a third-party tool and entering their TradeMe details. But, this has been solved many times, so I’m sure we could work out a way!

    As an aside, I think the feed for the new TradeMe gadget is actually exposing more information than is currently available on the website. Now you can look up a person by username, I don’t think you could do that before… At last I can see who got to ‘kirk’ as a username before I got in there!

    Kirk

  4. I’d agree with Nat on this. The major threat to the TM dominance is that the auction data could be presented alongside others (a NZ auction meta-search). I’d be willing to bet that if you did have an API including search I could do a profitable app by offering auction-sniping, better searching, historical value searches and total NZ coverage. This would allow the growth of better niche auction sites (e.g. stamps) that currently don’t get enough eyes and therefore the market is distorted.

    An auction meta-search isn’t a new idea, and I think it has been tried before (cue sucking sound of corporate lawyers). I’d be willing to bet that with a determined/bankrolled competitor it could be done now by using a foreign distributed spider combined with a legal challenge to those quoted clauses — TM does allow some spiders…and TM results in google are almost search spam given they’re inevitably out of date. This is one spider you’re obviously keen to get (and SEO’d your URLs to get it).

    Don’t get me wrong, I agree you’re allowed to do whatever you want with the information in your site and as a business you must protect your position and shareholder value. I just think you’re being inconsistent and you’ll have to reconcile the strategic concerns before you can even think about widening your API access.

    PS Note that your T&C’s don’t correspond to your robots.txt…

    http://www.trademe.co.nz/robots.txt

  5. I can understand the strategic arguments (and the counter-ones too) but your last point is the most compelling, i.e. is it worth it versus your alternative uses for your time and money?

    Too many things are done simply because of the strange group think that I call internet socialism. Do it – or don’t do it – because it’s the right decision for your business.

  6. I can think of one compelling reason why you’d want an API. To fail to do so is to generate competition.

    Aside from the size of the market, TradeMe have been working on auction systems for quite some time. Your gear is reliable, easy to use etc etc. But there are a number of products for which some level of external integration is desirable, for example imagine if I wanted to auction off license plates (as an example, ignore legalities etc for the moment).

    At the moment it’s mostly possible to automate the process, I can generate an email at the conclusion of the auction to the buyer, who can then visit a link to complete a payment process etc, however I have no firm control over the listings – I’d have to manually keep logging in to TradeMe to update things like new plates that have become available, sales that never closed and whether I’m going to relist them or try on another forum etc.

    The amount of manual work involved in being the “interface” between my own systems and yours is so high that it is tempting to simply build my own auction gear in and ignore you entirely.

    While you may be laughing due to the size of your userbase, in the right niche that’s irrelevant, if I’m already known in the market for selling plates new, and I then add a second-hand auction system, I will capture a reasonable chunk of that market…and it has just become that much harder for you to operate in it.

    An API gets the best of both worlds. I don’t have to invest time and effort creating a (likely crappy) auction system, and you don’t find yourself getting cut out of areas of the market because you won’t provide people with an API to work against.

    You also minimise the risk that someone will turn up with a solid API and steal a pile of those niche markets all at once, because in those areas their userbase doesn’t matter and all that really matters is the quality of the implementation, something that is doable.

    Whether anyone will ever do that in NZ is hard to say, but I believe there’s a good business case for taking APIs seriously, at least for that particular problem.

    Regarding the various downsides you listed, you need to differentiate between having an API for people to develop applications for customers against, vs an API for people to integrate their sales process. The two have crossover but they’re not the same.

    If I wanted to create, say, a firefox plugin or a list-manager website to let me list things, obviously there are serious implications for the Trust thing.

    On the other hand if I’m trying to get my license plate ordering system to integrate with TradeMe to list new plates, it’s my gear working with yours direct with regard to my account.

    If you’re worried about abuse, require people to get an API key. If you’re worried about future-proofing, keep it really really simple. You don’t have to provide much, just enough to stop people having to go and do stuff manually when it is so inefficient to do so that it would compel them to write their own gear instead.

  7. I agree with Jim. TradeMe is not a charity and therefore what it does should be based around what works for it.

    However I believe that satisfying the user needs always works for the business that concentrates on it – look at eBay, Google and even your goodselves with the interface.

    What about a forum/place to let developers make suggestions, ask for API’s and see if there is a demand – maybe no-one wants to, maybe everyone wants to.

    Oh, and RSS – didn’t I read somewhere that you could now subscribe to a traders listings … or was that something else?

  8. Hi Rowan,

    I feel you’re looking at the situation backwards. You’re speculating that by offering an API, you’ll be hurting your chances of entering the software tools market, cause increased load to your infrastructure by use of badly designed or inefficient software or cause backlash when you have to alter the API to add or change a feature, thereby breaking software that uses it.

    The thing is – this is already happening! There is already listing software that larger sellers are absolutely dependant on (Auctionitis, TradeTool and other custom software). This software has no choice but to use inefficient screen-scraping techniques to perform their tasks, thereby putting more load on your servers needlessly. Plus, when these tools break because you had to change the layout of a page for usability purposes, users will be quick to think: “There goes Trade Me breaking things again!”

    You say Trade Me values the user experience which is great to hear, but currently you’re forcing your biggest customers (think sellers with 1000+ concurrent listings) to use unsupported software (in violation with the T&C) to use the site efficiently. This doesn’t sound like a great user experience to me.

  9. Bwooce: Don’t forget that Trade Me is a marketplace where buyers meet sellers. You can build an API for sellers without building one for buyers. This is what eBay did, in fact. They tied their API very directly to their bottom line, and it paid off–as I said, 40% of their listings come from the API as people use specialized third-party clients to launch and monitor their listings and customers.

    Sniping, meta-auctions, etc. all use buyer-side data. Yes, they are risks but they’re also not necessarily world-enders. Take meta-auctions, for example–it remains to be seen whether they really are more of a threat than they are an opportunity. Trade Me makes its money from listings, and the more listings the better. eBay found that they ran into a cap on growth by only serving sellers–eventually they had to launch APIs for buyers simply to increase throughput. Only by having more transactions complete (and thus sellers going “holy shit, what else can I sell?”) could they deliver new growth.

  10. I forgot to mention: the ecosystem of third-party developers, which you’re viewing as a liability, are actually an asset. These are people committed to making sure that Trade Me stays the best place to run auctions–their customers are your customers, and the tool builders are committed to your success. If you aren’t the #1 auction site, they’re out of business too.

    The whole “other people might benefit”, “take no risks, we’ll get the rewards slowly without the risk”, “we’ll do it all ourselves” mentality is driven by fear and paranoia. That doesn’t seem like the Trade Me way.

  11. I’d still do Travel first. And then X, and after that Y. whatever they are. Surely there is more benefit there.

    An API is, as Nathan points out, firstly for sellers to help get listings on to the site.

    Trade Me has 865,000 listings right now, while eBay Australia has just 1,096,000 listings.

    Australia has a population of 20.2m and NZ 4.1m.

    That’s 0.21 listings per Kiwi and 0.05 listings per Aussie.

    eBay demands listing fees, and that tends to depress listing numbers (by about half). But that doesn’t nearly account for the difference in listing numbers, and that difference pointedly shows that the lack of an API isn’t holding back listings numbers.

    eBay.com itself has 10.5m (0r 11.9m) items located in the USA. That’s 0.035 items per person there. That may exclude store inventory, and it seems really low, and still – pathetic versus Trade Me.

    Clearly whatever Trade Me is doing to get listings is superior to eBay. Vastly superior.

    Hold the API I say.

  12. Hey Rowan,

    I, as I am sure a lot of dev’s in NZ, have a few great ideas which could make use of an API and make valuable tools for your users. I could only see this as a mutually beneficial relationship in that if I do well, you do well. And if I abuse the API, my API key gets revoked.

    Failure to provide API’s and external data access has lead to other sites gaining significantly on you in such areas as real estate (only my personal opinion as I am looking to buy a house). While these sites and other emerging markets might not have overtaken you, innovation and greater exposure will keep you ahead of the game. It might not make sense to get your small group of internal developers to do this but an external one using an API ……

    I fully understand the reasons why TradeMe has the stance that it does, but I believe with careful considerations and implementation it could really be a win win situation.

  13. ‘An unrelenting focus on the user’ – Sam Morgan (NetGuide Awards 2006).

    I would be interested to know how Trade Me makes decisions about implementing new features. Trade Me clearly limits usability in some instances (not being able to contact a seller to view an item before buying). In other instances they are one of the few sites to hold out for the users (no Flash banner ads).

    The API I see as a few feature for sellers. Integration with 3rd party listing software and stock management tools seems like a must-have feature for any serious seller. This will be especially relevant as more new goods get sold online.

  14. Main reason I would like to see API is so that I could see accounting package like US version of microsoft accounting 2007 which can use ebay API.

  15. Hi, I have been observing TradeMe auction for a very long time now; and notice lots of the auctions have the following line ‘[ Loaded by Auctionitis ]’. Driven by further curiousity, I try to look at Auctionitis website (http://www.auctionitis.co.nz/) and oh boy… $120 for single license, with a minimal support, and no printed documentation? And by July 1st, 2007 they will increase their price…
    I tried to find any alternative for Auctionitis at google, but haven’t got any luck yet.
    Reading your argument here make me come to a conclusion that those guys at TradeMe are just trying to create a monopoly. You are not publishing any API because you don’t want to have any competition for Auctionitis.
    You argue that you encourage people not to sign-in in any place except TradeMe web site, but you authorize Auctionitis as a tool in ‘Trade Me Success Secret’ book? Well, that’s sounds like hypocrisy for me.
    So, when can we see TradeMe publish the APIs and level the game field?

  16. I just want to re-touch on something Bwooce said as i too find this quite interesting.

    *Trademe Clause 4.1c*
    “You may not use a robot, spider, scraper or other automated means to access the Website or information featured on it for any purpose.”

    So, theoretically (as bwooce mentioned) google, yahoo and every other search engine in the world that has spidered (and indexed!) trademes site is in violation of these rules.

    Also, my understanding is that a browser is a piece of software that takes HTML (plain text) / scrapes it / renders it and provides gui widgets to the user that automate or make easier the process of sending data back to the server (plain text). For example combo boxes / list boxes buttons etc. It is also my understanding that whilst there are web standards as to how a browser should render these pages there is no laws to enforce such rendering.

    Of course trademe could write into their rules and regulations which browser types were allowed to be used but that too wold be a weak argument in court.

    With these points in mind, I have *a friend* who has a project where he has taken the mozilla firefox browser and successfuly rearranged it (with out modifying any source code) so It is 100% still a browser as we know it but looks nothing like it. For example main window shows a list of all the auctions and your account bal. Has a wizard too add listings and stores all the data in sqlite database (sqlite extension is part of mozilla).

    Its similar to auctiitis in that it it *eases* listing items on trademe, albeit the interface is less gadgety. It also keeps a database of all the listings and automatically compresses and optimises large imag files in the background before uploading. It can periodically post items from an inventory and track and respond to buyers emails.

Comments are closed.