883 posts Envato Studio Communications Manager
  • Member of the Envato Team
  • Attended one of our Meetups around the world
  • Featured in the Envato Market Community podcast
  • Has been part of the Envato Community for over 5 years
+7 more
Jordan_M Envato team says

Originally Posted by Vahid Ta’eed on Envato Notes

State of the MarketplacesA little while ago, we promised a post about our development roadmap, as well as what has been worked on in the last year. There is a laundry list of excellent features and updates that have been waiting in our pipeline for what feels like forever. And understandably there’s a lot of questioning on the forums over why they haven’t materialized yet.

Search improvements, ratings improvements, a shopping cart, item support tools, affiliate campaigns and analytics, license improvements and the list goes on. It’s a list of things that would make buying better, selling better, and alleviate problems left, right and center.

The long and the short is that a lot of these types of improvements keep getting put on hold as we cope with growth and expansion.

Growth and Expansion

The growth on our marketplaces has been breakneck over the last few years with everything from revenue to user count to transaction volume to item count to fraud attempts to security hacks doubling, tripling, quadrupling or more year on year. To be honest I find a lot of the numbers kind of staggering.

On top of that, we’ve historically pushed the team very hard to expand into more and more niches, last year’s PhotoDune release being by far the largest. This is an important part of Envato’s marketplace strategy and is what has gotten us to where we are today. But it has the adverse effect of multiplying all our problems and making solving things harder because of all the use cases. So for instance changing licensing needs to change licensing for many different types of items, review queue upgrades have to take into account many different review scenarios, and so on.

And finally though we’ve been trying to expand our dev team for a little over two years now, we’ve had a lot of difficulty doing so. The hiring market has not been our friend on this front. The tide seems to now be turning and we’ve increased up from 7 to 9 marketplace devs in the last couple of months, with a tenth joining in the next few weeks. And we have job ads (http://techjobs.envato.com/)¬†and recruiters out to increase our team size even more dramatically.

Scaling

On the scaling front, I checked in with our lead architect Pete who tells me we now have 8 physical machines with 68 cores and 432 GB of RAM , plus 17 virtual machines including 9 front-end servers, and we are serving up some 150 TB of traffic a month. In short, it’s a truckload of horsepower, and just scaling it all is a huge load of work. But it’s paid off, despite the enormous growth in traffic, you will notice that our sites are pretty snappy and rarely run into downtime. Security and fraud also take up quite a bit of time, and the bigger we get the more of an issue they become. Our level of response sophistication keeps rising as the threat level does. We are coping with both the scaling and security and fraud challenges, but it is a large amount of development work every month!

Expansion

Last year almost 50% of our dev effort went into PhotoDune. It was a mammoth project because the profile of the marketplace is so different to our other sites. However at the beginning it wasn’t clear to me just how much resources it was going to take. And those resources are the same ones that would have gone on to the laundry list of features I mentioned earlier.

I’m not sure whether it was a bad call on our part to push for PhotoDune, I go back and forth about it. But it’s a call we made, and now we have a kick ass photo marketplace which means generally happier buyers, more resources for authors to draw on, and more potential for cross promotion.

But if you want to blame one thing for why we haven’t been able to deliver on many of our promises of improvements, you can put it the decision to build PhotoDune!

Search & Review

Two other areas we spent a lot of our time on last year (14% and 8% respectively) were Search and Review Queue.

The search project is an ongoing one. The time spent last year was essentially to change our search server to Solr. As of yet the benefits of this change aren’t apparent on the front-end. That’s the next portion of the search project. It’s always tough having a project that is long, complex, important, but with no visible improvement! But I’m very confident that we will reap huge benefits in the near future from the work last year.

The review queue was another big project which is still underway, and once again completely invisible to the customer. Because we are dealing with increasingly large volumes of increasingly different items being reviewed by an increasingly large review team … we had to completely overhaul our review queue system. It started with PhotoDune, and is slowly migrating out to the other sites.

The difficulty for both these projects is that there’s an interim period where you have to maintain both the old system (whether Sphinx old search or old review) at the same time as rolling out the new system (Solr search and new review queue). This just increases the level of dev work needed as you are pulled across two blocks of code in each case.

The “Maintenance Queue”

Pretty much all the front-facing changes and updates you would have seen in the last year come from the maintenance queue. This is our queue of little fixes and changes. It’s anything which takes a half day to a couple days of a single developer. At any one time there are two devs allocated to the maintenance queue.

It’s an important queue because it’s the one that deals with bug fixes, attribute updates, and a variety of miscellaneous tasks both front and back facing. However it’s the minority of the total dev work.

While it’s frustrating seeing a new badge roll out when much more pressing stuff like ratings improvements are waiting, it’s important to realize that the scale of these changes is like a pea and a porsche. It takes next to no time to roll out a new badge, so you may as well keep tasks like that running in the maintenance queue.

Stopping those sorts of rollouts wouldn’t actually make any difference to the big ticket items. Stopping the entire maintenance queue would mean being able to do another project in parallel, but the consequences of not tending to bug fixes, minor improvements, performance tweaks and so on, would be pretty bad. We’ve done it in the past for short periods, but the pain escalates fast.

What’s Ahead for the Next 6 Months

The good news is we are expanding our dev team and resources quickly now. The bad news is expanding means turning parts of the codebase into separate services that are connected through interfaces. If that sounds like a dev project, that’s because it is! We have several such server projects in the pipeline for the next six months. They will be long term wins so that we can deliver new features while growing, but short term more internal facing work unfortunately.

We’re also finishing off the review queue project, and there’s a big and pressing finance project in the works too. The outcome of both these two projects is that the marketplaces continue to work and churn out lots of cash to authors!

The big front-facing project is the other side of all the search work we’ve done last year. It’s the largest single project we will be undertaking in this next half year period. And I’m really excited as it’s something that will be a huge win for buyers, which in turn is a win for authors.

Realistically in the next 6 months those will be the main things to come out of the marketplaces. It’s not everything I would wish for, but they are all critical projects.

Ultimately when it comes to marketplace development, we’re playing the long game. And at the moment that is pretty frustrating all around. But it will come good, and along the way the marketplaces will just keep getting bigger and bigger!

We’ve put a hold on new marketplace expansion. And we’re going to stop promising features as being coming up soon, or in the pipeline. Instead we’re aiming for more transparency (like this post) to keep the community abreast with the current challenges we are facing and how we are dealing with them.

I hope that this post is helpful, if you have ideas on how it could be better, let me know in the comments. Similarly if you have questions or thoughts, add them in!

737 posts
  • Has referred 500+ members
  • Has sold $125,000+ on Envato Market
  • Has collected 50+ items on Envato Market
  • Elite Author: Sold more than $75,000 on Envato Market
+7 more
mordauk says

Thanks for the updates, Jordan!

1449 posts Determined to give you nothing but the best!
  • Has sold $125,000+ on Envato Market
  • Elite Author: Sold more than $75,000 on Envato Market
  • Made it to the Authors' Hall of Fame
  • Had an item featured on Envato Market
+9 more
mrcharlesbrown says

Thanks for the post Jordan!

1055 posts Best-dressed man at PressNomics 2013
  • Has sold $1M+ on Envato Market and is now a Power Elite Author
  • Power Elite Author: Sold more than $1M on Envato Market
  • Made it to the Authors' Hall of Fame
  • Had an item featured on Envato Market
+10 more
Parallelus says

Thanks for the detailed information. Whatever you did to streamline the Review Queue has worked. They’re approved so fast I’ve been caught unprepared once or twice making tiny tweaks to my demo because I thought I’d have more time. :)

1040 posts Freelance Flash and JavaScript Developer
  • Has been part of the Envato Community for over 8 years
  • Elite Author: Sold more than $75,000 on Envato Market
  • Has sold $125,000+ on Envato Market
  • Made it to the Authors' Hall of Fame
+7 more
sike says

Thanks for the hard working. ;)

2485 posts Premium Scripts, Plugins and Themes
  • Has been part of the Envato Community for over 7 years
  • Has referred 200+ members
  • Has sold $125,000+ on Envato Market
  • Has been a beta tester for an Envato feature
+6 more
LucidStudios says

Thanks for the updates. Keep up the good work.

8134 posts
  • Has been part of the Envato Community for over 5 years
  • Has sold $40,000+ on Envato Market
  • Sells items exclusively on Envato Market
  • Has referred 10+ members
+4 more
doru says

Search feature is critical, I hope it was designed well and it will take in account the different types of files available, like for example searching for music on audiojungle and show only those results that have a certain time duration.

2485 posts Premium Scripts, Plugins and Themes
  • Has been part of the Envato Community for over 7 years
  • Has referred 200+ members
  • Has sold $125,000+ on Envato Market
  • Has been a beta tester for an Envato feature
+6 more
LucidStudios says

Search feature is critical, I hope it was designed well and it will take in account the different types of files available, like for example searching for music on audiojungle and show only those results that have a certain time duration.

I agree, the search feature is very critical and should be very robust. If I search term “calendar” in codecanyon marketplace then it shows items at the top which are not even calendars this problem should be solved.

Also I would like to know how the ranking of items in search results work because I have seen a lot of fluctuations for my only codecanyon item and now it has moved 2nd page despite having relatively fair amount of sales.

684 posts
  • Has been part of the Envato Community for over 2 years
  • Has referred 10+ members
  • Sells items exclusively on Envato Market
  • Located in Indonesia
Gembell says

Yeah, thanks for all . . .

337 posts No! It's not a 5 minute job!
  • Elite Author: Sold more than $75,000 on Envato Market
  • Won a competition
  • Has sold $250,000+ on Envato Market
  • Had an item featured in an Envato Bundle
+13 more
WebFactory says

Thanks for the info!

Helpful Information

  • Please read our community guidelines. Self promotion and discussion of piracy is not allowed.
  • Open a support ticket if you would like specific help with your account, deposits or purchases.
  • Item Support by authors is optional and may vary. Please see the Support tab on each item page.

Most of all, enjoy your time here. Thank you for being a valued Envato community member.

Post Reply

Format your entry with some basic HTML. Read the Full Details, or here is a refresher:

<strong></strong> to make things bold
<em></em> to emphasize
<ul><li> or <ol><li> to make lists
<h3> or <h4> to make headings
<pre></pre> for code blocks
<code></code> for a few words of code
<a></a> for links
<img> to paste in an image (it'll need to be hosted somewhere else though)
<blockquote></blockquote> to quote somebody

:grin: :shocked: :cry: Complete List of Smiley Codes

by
by
by
by
by
by