Wildcard launched our first product a few weeks ago – a card-based mobile browser. It’s a new way of browsing the internet on your smart phone, and it’s just the beginning. A group of incredibly talented people came together, designed and built the app over the past 12 months. I want to share a few things under the hood for those who are curious.

Browser

We started our journey with the goal of a thin client on iOS. Instead of making a runtime that interprets programming languages and specification of behaviors, we opted to build an app that recognizes “types” of cards and focus on the data. For example, the browser understands the a “product” card, or an “article” card. We did this because of the speed of implementation, and the risk of increasing complexity for a “card language”.

The effort involved, however, is far from the sound of a “thin client”. It’s carefully designed and built by our product / iOS team. Significant efforts has gone into things like custom animation, custom gesture, local caching, etc.

The Wildcard browser is written in Objective-C. We chose this because of implementation speed and user experience. Cross-compiling technologies like PhoneGap or RubyMotion break down quickly when you want to have custom UX. Even though this means we have to start from scratch for Android, I expect the Android UX to be quite different that code-sharing would be very difficult regardless.

Platform

When we started our journey, the mobile card ecosystem was in its nascent state. We started by decomposing webpages into their most essential components – which eventually evolved into our own concept of “mobile cards”. A platform was created to convert generic webpages into cards.

For example, a product page on Lululemon might have a search bar, the product details, a checkout button, and a locator to find the closest store. We would create a “product search” card, a “product” card that allows users to checkout, and a store locator card.

Many iterations have gone into the platform, and it has evolved into a framework to execute commands like “extract image url at this XPATH” or “Parse this piece of string into a proper U.S Address”. Many of the iterations have to do with getting to the right level of abstraction, so that humans can manipulate the data pipeline as well as computers (through layers of abstraction). These commands also became building blocks for more complex algorithms – and we are working on machine learning techniques to make the platform more robust.

The platform acts as a blackbox that turns inputs like an URL or a piece of structured data, into a mobile card. It’s configurable by humans and algorithms, it’s parallelizable across machines, and it’s at the core of our technical infrastructure. We chose Java and Dropwizard for a good trade off between performance and ease of implementation. The renaissance in the JVM stack gives us confidence in our technology choice.

Data Services

Wildcard works on many large-scale data problems. As the complexity of our system grew, we grouped together services that directly manipulate data. This set of services includes things like RSS feed readers, small web crawlers, one-off scripts for data processing, and GUI tools to allow direct data manipulation. This is a high-touch/offline environment, where a lot of “raw data” gets processed into a state that can be fed into the platform through a simple API.

Having an offline environment gave us the ability to move quickly without worrying about the stability of our platform. Things can get a little messy if you are not careful, so following good software engineering practices is extra important here. We use a mixture of different technologies in this system, with the dominant languages being Ruby and Javascript.

Search Engine

As we built more and more cards, we started to realize the importance of search. Search is often the best way to expose a large variety of experiences to users. In our case, it has become an essential component of our product.

Building a search engine from scratch is no easy task. We started out with a vertical approach – focusing on e-commerce first. We used techniques like LSA to map queries onto commerce brands, rank the brands, and automatically kick off search within the brands. This is a powerful feature because users will always get the most up-to-date search results from the brands, and we wouldn’t have to do a massive/costly web crawl.

Since then, we have added many more result types into our search index. We have also built a scalable and configurable crawler (across the brands that use mobile cards). Search is the fastest evolving service in our stack, and we are constantly trying to expand our capacity / use cases.

We currently use a combination of Redis, Elasticsearch and different python scripts for search. As the index sizes continue to grow, more scalable solutions will be needed.

Integration Systems

One of the goals of Wildcard is to make our mobile card platform “writable”. This means any content publisher, retailer, service provider is able to take advantage of the system and control their own cards. To make this as easy as possible, we’ve created a product just for that. It allows anyone to publish to Wildcard, Twitter and Pinterest.

We built a schema translation layer in Java to support this product. We also built tools like this to help developers validate and visualize their own card implementations.

Dev Ops

As our engineering team grew, automation became more and more important. We started out with using Chef, Jenkins on top of AWS, and have since adapted an concoction of automation tools.

At the core, we use AWS as the base layer of our infrastructure. We use Chef for server configuration, terraform for orchestration, docker for resource management, graphite/keen for metrics and monitoring, consul for async jobs (like deployment), github/circleci for continuous integration, hubot to tie everything together, and probably many more tools I forgot to mention.

Automation has been a game changer in our daily engineering lives. It’s a significant investment – and it’s necessary. When done well, it can be a significant productivity booster for the engineering team.

As our 15-person product/engineering team continue to grow, our technology will continue to evolve and adapt to new needs. If you are interested in anything discussed here, shoot me a line and I’m happy to chat more.

Just a short 14 months ago, I was faced with a decision of joining a startup as the first employee. Having worked at a slightly bigger startup, I have had some experience with working in small teams. But the decision of joining a team of 2 people is still a tough one. It was a leap of faith, facilitated by some gentle persuasion by Jordan and Doug. Late last week, we announced that Hyperpublic is acquired by Groupon. I can’t say I knew this was going to happen 14 months ago, but I know that I’ve made the right decision. If you are in a similar situation, I hope this post will help you rationalize your decision making process.

People join startups for different reasons, but almost everyone does it partially for the upside of making lots of money through the equity plans. This is probably the biggest myth for startup employees. The chance of your startup turning into Google or Facebook is practically zero. So the first question you should ask is:

Why should I join a startup if it’s not for the money?

This means completely rule out the money factor, and this is where people’s opinions diverge. For some people it’s the thrill of building something from the ground up, for some people it’s the prospect of taking on a wide range of responsibilities. Whatever it is for you, this is not a matter to be taken lightly. Here are some questions I see people often have and my thoughts about them:

Q: I want to get in early so when this company goes big I’ll be on top. Should I join a startup?
A: If this is your main motivation, it’ll be a long and treacherous process for you. The benefit is far out-weighted by the risk here, you’d have to be a really irrational person to do it. I find the carrot is usually not enough incentive to carry people over the hump. You will be exposed the ugly truths as the company figures out a path to survival, and it’s sometimes soul-crushingly painful. At times you will feel like you’ve just wasted the last 6 month, year, 2 years making something that is doomed to fail, and your friends who work at the big-co next door is making 150k salary while cruising through life. And you will get discouraged. But unlike the founders, you are not nearly as invested in the mission, so you will quit.

Q: I’m not sure about the founders, but I love the mission of this startup. Should I join?
A: From my experience, the is almost a sure death. As proven by history time and time again, ideas aren’t worth much. Ten years ago, people use to value ideas alot more, partially because the startup culture back then was less open. But today startups have become more agile, and the startup landscape is changing more rapidly than ever. Don’t join someone you can’t completely respect.

Q: I don’t know a lot about this industry, but I’m super impressed by the founders. Should I join?
A: Opposite of the last one, and much more promising. Sometimes it’s difficult to evaluate the quality of an idea, especially if you haven’t been working in that industry for more than a few years. Valuable companies are the ones able to adapt and execute, which makes the original idea less valuable. This puts more emphasis on the founders. How do you evaluate a founder? A great track record helps, but not having it doesn’t mean they cannot succeed. They have to be intelligent, genuine, and most importantly, super resourceful. If you find someone who check off all of these marks, I’d say go for it. Good founders have the tendency of getting stuff done come hell or high water. Even if the startup ends up failing, having worked with great people leads you to other great opportunities.

Q: I’m ready to move on, but my team now needs me. Plus, my boss would kill me if I leave.
A: We’ve all been there. You don’t want to let your teammates down by leaving. But this is a 2-way street. This is what happens if current job is not doing it for you and you try to be a “nice guy” by staying: you still don’t enjoy working there, your current work will go as it’s always been, and the startup will suffer. No one really wins in this situation. Give your 2-weeks notice, be respectful, and move on.

Q: I feel like I’ve worked in the corporate environment, it’s time to move onto a new challenge with more responsibilities. Should I join?
Q: I want to start my own company some day, so I’m here to learn. Should I join?
A: These are what I consider “the right reasons” to join a startup. To me, startup is a lifestyle. It’s really exciting at times, pretty fun and interesting most of the time, and really stressful and devastating occasionally. What carries you through the tough times is your personal resilience, passion and curiosity. You will get the opportunity of doing a little bit of everything, taking something by the horn and running with it. You get rewarded for taking risks and learn from making mistakes. The right attitude here takes you a long way.

Q: I can always go work at another startup if this one doesn’t work out right?
A: True. But you want to be careful here. Think of joining a startup as a 2-3 year career investment. How many 2-3 year chunks do you have? How many are you willing to invest?

Here are some day-to-day stuff to think about:

Q: I hear startup people work crazy hours.
A: That’s true somewhere. But if a company is well-ran, you shouldn’t be working more than 65 hrs a week on average. I’d say I worked a 60 hr week on average at Hyperpublic.

Q: I’ve made a lot of friends at my old job. Will I do the same at a startup?
A: It really depends on the company. Some companies are really social and some keep it professional. If you really like a social work environment, you’ve gotta make sure they have that.

Q: I hear you meet a lot of cool people (outside of your own company) by working at a startup. Is that true?
A: It’s true, but it doesn’t come for free. There are plenty of “networking” events, going to all of them can suck up a lot of time. You don’t want to talk to the people who are JUST NETWORKING. My advice is to pick events hosted by good startups, and pick the ones that will teach you something. Getting out of a conversation politely is a good skill to learn. Remember to always be respectful.

Q: I’m a PHD in (Machine Learning | NLP | PL | Cryptography| etc…), what’s it gonna be like for me at a startup?
A: Pretty different than your academic research life for sure. I don’t have a PHD but I’ve done a few years of academic research and have worked with many PHDs both in research settings and startup settings. Sometimes you are gonna have to write that simple CRUD app or that piece of front-end javascript just because everyone else is busy and the company really needs it. But you’ll also have a lot more control in the work you do, and that’s utilizing science in a truly practical manner.

Q: What’s the most exciting part about joining a startup?
A: I can only speak from my own experience. The most exciting thing is that I’ve grown so much faster in the past 14 months. I find myself becoming more resourceful. I have more ideas when it comes to getting stuff done, and I learned it through getting stuff done everyday in the last 14 months.

Q: What’s the suckiest part about joining a startup?
A: As a startup, there is no room for HR mistakes. That’s why when there is a misfit, you have to execute quickly. This is not personal, and it’s not even a statement on the person’s ability. It’s simply a “misfit”. But it still sucks to see people who have become your close friends being let go.

Q: Just how much WILL I make if all goes well?
A: Let’s just do a back-of-the-envelope calculation: let’s say you get 3% of the company (This is completely arbitrary. See  PG’s essay for an idea on how much you should ask for). After a few rounds of funding (and most likely 2-3 years), you will be diluted to less than 2%. This means if your company gets sold at 100million, you will make less than 2million. Think about how many companies you know that’s sold for 100million, then think about how many launch announcements you hear per week. I think you get the picture. Also see here and here

Q: I’ve considered everything. Still not sure about this.
A: Try joining on trial basis. This is a great idea for both the startup and the employee if executed correctly. If you decide to do this, do some research about the startup and make sure they don’t have a bad track record of trying to get “cheap” work out of people this way. Do it on a project-basis and set the terms for both cases (hire or no hire) before you start.

Conclusion

Each of these points can be its own blog post, and I might pick specific topics to go into depth in the future. Good luck with this decision. It can be the best thing you’ll ever do for your career.

I’ve wanted to play with Jekyll for a while now. A delayed flight and awesome internet connection at the Pittsburgh airport finally presented me the golden opportunity.

Here is a step-by-step tutorial on how I build this site:

Install Jekyll

You should first install jekyll. There is a good tutorial on the Github wiki. But If you don’t want to follow the official tutorial, here is what I did.

Install the Jekyll gem by running:

gem install jekyll

If you want syntax highlighting for code blocks, you have to install a python module called Pygments.

sudo easy_install Pygments

Of course you have to have python-setuptools installed.

First blog entry

After installing jekyll, I created my site structure that looks like this:

      .
      |-- _config.yml
      |-- _includes
      |-- _layouts
      |   |-- default.html
      |   `-- post.html
      |-- _posts
      |   |-- 2012-01-20-some-post-I-wrote.textile
      |   `-- 2012-01-21-how-I-built-this-blog.textile
      |-- _site
      `-- index.html

The best way to get started is to take a look at someone else’s project. Since I’m terrible at anything css/html related, I borrowed a lot of code from User Obsessed.

The _config.yml file contains configuration settings of the site. Here is what mine looks like:

url: "http://www.erictang.org"
    pygments: true
    exclude: [/bin, Gemfile, Gemfile.lock]
    markdown: kramdown
    permalink: pretty
    paginate: 10

    kramdown:
      entity_output: symbolic

The index.html file is your homepage. The layouts directory contains the formating of your pages, and the posts directory contains individual posts.

To get a simple site up and running, I created config.yml, index.html, layouts/default.html and _posts/2012-1-21-sample-post.textile

After that, run

jekyll --auto --server

You should see a server starting up. Go to http://localhost:4000 to make sure it’s working.

There are many other things you can do with this barebone site, refer to the Wiki for details.

Disqus

Comments are important for blogs. There are many different ways to add comments to your Jekyll-powered blog, I chose to use Disqus because it’s easy to integrate.

Luckily Disqus has make this process SUPER easy. Here is what you have to do:

• Sign up for a disqus account, register your domain.
• Copy the embeded code to your _layouts/posts.html. Remember to replace “example” with your own shortname.
• Reload the page.

That’s it!

That’s it! You’ve set up your first jekyll blog!