Stritar's chronolog

5 reasons why I won't steal your idea

Tue, 05 Feb 2013 09:22:39 GMT

Since I'm a software architect and a web developer, I get often approached by people with their new ideas. In most cases, for some quality feedback, and on lucky days, for a rough quote about the costs of such a project. These people are usually very secretive about what they have, making me explain to them that it's far from my interest to steal that idea. One time, a guy even made me sign a Non-disclosure agreement before I could make him an offer for a service he was thinking about. After bargaining with me, he chose a different contractor, but ended up doing nothing, at least to my knowledge. He was obviously focused on the wrong things, instead of getting feedback from as many sources as possible, he was investing energy into bureaucracy and protection of his idea. Let me tell something to him and all others out there: Focus on your product, and don't worry about me stealing your idea. I won't. I have at least five reasons not to.

1. Your idea probably isn't as great as you think

I've seen a lot of different people who had "game-changing" ideas, at least so they though. A few of them actually managed to convince me and my partners that their idea is so amazing that it'll kick everyone's ass. Even though proper market research wasn't done, charisma is sometimes hard to resist, and if you are working with someone you've known for a long time, you are prepared to accept crazy terms, such as a delay of payment until this idea will start to generate revenue. After these specific ideas were put into the real world, it turned out there is a huge discrepancy between ideals and reality, and we ended up with unpaid invoices and ignored phone calls.

Ideas are something, execution is everything else. There is a long way inbetween, a way paved with upgrades, downgrades, changes, pivots, time and hard work. Millions have ideas, only a few can make them work. I've seen Seedcamp companies change their core concepts and business models completely, and these startups are already the best, selected from hundreds, if not thousands. When you start working on something and proceed ahead, the initial idea will often evolve beyond recognition. Not to mention there is a very strong possibility that someone else was already thinking about the same thing, except better, years earlier. Your idea isn't amazing, but it may be good enough to achieve something with proper execution. That's why you need feedback and partners.

2. I have plenty of ideas of my own

You know what's better than your idea? My idea!

Mostly because I kick ass, but also because people get emotionally attached to the thoughts they generate by themselves. I have so many ideas I don't know what to do with them. They are probably not really great (see reason 1), but they are mine, and I try hard to make a few of them come alive every now and then, when I have the time. During the day, I work on real-life projects, during the night, I play around.

When I decide on what to work on next, I usually look for the best ratio between actuality, complexity, required energy and potential. This means I've already made my own priority list of the services I will be rolling out in the future, and I must say, it would really be hard to put one of yours inside this packed list. I'm sure most developers think in a similar fashion, lacking resources to make everything they imagine a reality. Face it, there are hundred times as many people who have unrealized ideas, than people who don't know what to work on. Do the math.

3. Your idea probably requires specific passion and know-how

The idea lives strongest in the person who thought of it. It is a result of that person's experience from many fields, so it's hard to replicate in its full form without that experience. The ideas I've stumbled upon usually solve very specific and niche problems you can't solve without digging yourself into that field. Which most of us don't have time or the resources to do. The core of the idea represents the person who thought of it, it may be taken to another level by a different person, but in most cases, it requires the original author's knowledge, involvement and passion to work as it's supposed to.

I can't execute an idea which will revolutionize kindergarten children education, I don't know shit about the problem, I don't have any connections in the industry, and I'm simply not that passionate about that field.

4. Your idea requires your involvement as a product manager

In the past few years, we've implemented quite a few prototypes and services together with Neolab, and even though a few of them got some praise, they didn't make it to the mainstream. Take this blog for example, I developed it in early 2009, aggregating posts from different social services, presenting them (also) in a magazine form. What did I do with it? Nothing. Years later, a service called RebelMouse did something similar and raised millions in funding. Same goes for Twenity, a spin-off from Twitfluence, gamifying social authority measuring. Or Ljubljana Realtime, a social event discovery tool. All out there, but that's it.

I simply don't have enough energy to push a service beyond a point, or don't want to. Perhaps this fact will change someday, but at this point, you will need to be the product manager of your idea, and I can be its architect. Since I have a real company to run besides all of this, I can't afford to be one.

(Btw, if you think you could do anything with the above mentioned things, don't hesitate to give me a shout).

5. Karma and stuff

I believe in karma. Don't do evil and all of that. I would really feel uncomfortable if I would take someone else's baby and make it my own. So I won't, because this simply wouldn't be a fair thing to do. I value proper sleep above success.

But what if

Of course, there are always exceptions, and I can imagine I could encounter something that would go beyond all of my points above. Perhaps there is one idea that I've heard about years ago that would suit this description. If I ever decide to proceed with this project, I will let that person know what I'm doing and invite him to join the project. Even if I'm thinking about a thing that only faintly resembles the original concept, I can't deny it's that person's idea. And since this guy was able to think of such a marvelous thing so much time ago, he would surely make a great addition to the team (also see reason 3).

That's it. I won't steal your idea, so feel free to talk about your revolutionary innovation with me anytime. All I will do is to try to tear it apart and put it back together, and after we're done with that, if I get the chance, I will try to bring it to life.

Using JSON (with asp.net) is like wiping your ass with silk

Sat, 05 Jan 2013 18:45:44 GMT

I love mashups. Actually, I love everything about them, I love using them, I love making them, I love those who do everything they can to empower them. In my opinion, mashups are one of the most significant concepts the Web has invented, since they represent unlimited possibilities of integrating and reshaping things that are already done. The platforms out there are stable, so it's the creativity that sets the limits. These days, you can easily take data from anyone and do something else with it. Just don't forget to use JSON.

I'm an old school guy who has been working mostly in enterprise software environments, which means using XML for data integrations, besides, RSS used to be the man. The first generation of Twenity (Twitfluence) was using XML feeds, and this blog is using mostly RSS for fetching entries from other sources. XML was the standard we all spoke, very cute and readable, but on the other hand, not so easy to parse. At that time, this fact didn't represent such a major problem, since a few more lines of code took care of everything. But today is a different situation.

It was Twitter who first started dropping support for XML, which annoyed the hell out of me. I had to start rewriting things for JSON. This turned out to be one of the best things that ever happened, since JSON seems to be loved by everyone. It's super easy to use with jQuery, but since I'm a server-side type of a guy, it's even more important that it's cleverly integrated into asp.net. Fetch the data, store it, do crazy shit with it.

JSON is serializable into a .net object with a single line of code. Create the class with parameters compliant with the the specific JSON structure, serialize the response string into that class, and everything automagically works. Piece of cake, unlimited opportunities. Take Twitter for example:

Make the basic Tweet class:

public class Tweet
{
public string id_str;
public string text;
}

Create the request to access a tweet (funny, the hardest thing to do):

string url = "http://api.twitter.com/1/statuses/show/274508827146215424.json";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = "GET";
WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string jsonResponse = reader.ReadToEnd();
reader.Close();

Which will return something like this (play here):

{
"created_at": "Fri Nov 30 13:42:59 +0000 2012",
"id_str": "274508827146215424",
"text": ""Facebook knows what we say, Google knows what we think",
"source": "web",
...
}

Serialize the response string into an object:

JavaScriptSerializer js = new JavaScriptSerializer();
Tweet tweet = new Tweet();
tweet = js.Deserialize<Tweet>(jsonResponse);
Response.Write(tweet.text);

Pretty neat. Since I've started using JSON, mashups have become easier than ever to make. With one of our latest projects, we've integrated our application with Twitter, Instagram, Foursquare and Flickr. For breakfast! XML may have its advantages, but for such things, JSON is simply the greatest. All hail the new lord!

A case study in agile development: the algorithm for Ljubljana Realtime's event discovery

Mon, 22 Oct 2012 20:01:16 GMT

When we started working on Ljubljana Realtime, we decided to approach it in an agile way. Amongst others, we wanted to use a few interesting lean concepts such as rapid development, Minimum Viable Product and the Build - Measure - Learn iterations. Less than two months later, the results are in, and they are very pleasing. The MVP in the shape of an activity map was developed in a few weeks, only to show there is a lot of social noise which will somehow need to be taken under control. But that's currently low priority, since the first pivot is already taking place, slowly shifting the focus from the rich map application towards an event discovery algorithm and stream. That's where I see the most potential of Ljubljana Realtime, and in the last weeks, that's where the most work was done.

Launch

Ljubljana Realtime event discovery engine uses Foursquare trending venues and geo-tagged posts from Twitter, Instagram and Flickr to discover what's happening in real life. At least 6 people checked-in on Foursquare or two different people tweeting or posting photos in a single hour could mean something is going on. These events are posted to Twitter and Facebook, with links to the posts. A few versions of this algorithm were already deployed, each one solving new problems, resulting in a few micro Build - Measure - Learn cycles in a single month.

Iteration 1: Foursquare, no duplicates

The first version of the stream (bot) was a simple one, at that point it was meant to work as promotion for the map. The only thing it knew how to do was to wait a few hours until it posted the same thing again. I think Foursquare checkins are alive for three hours, so if a trending venue was still trending after that time, new people had to checkin and the venue was still buzzing.

Problem: Plain, no real added value.

Iteration 2: Adding activity from other sources

When we were trying to make some space on the crowded map, we started grouping posts from Twitter and Instagram by the nearest Foursquare venue, which meant having less boxes on the screen. This turned out to be quite a complex thing to do properly, but it was worth the effort. On only a few occasions, one venue would have multiple posts in a single hour, and in most cases, that meant something was happening there. This provided another very interesting potential for the activity stream. Actually, it made the stream bigger than the map could ever be.

(I love it when such things happen, when you are trying to solve a problem, and it turns out there is much more hidden behind the resolution.)

Groupping posts by a venue. Did Ljubljana Realtime just discover an athletic meeting taking place?

The next problem: Activity in some venues, specially generic ones such as "Ljubljana" would trigger the stream almost every day. Similarly, some large venues, such as supermarkets, would be trending too many times on Foursquare.

Iteration 3: Balancing the posts

The algorithm needed an update, which would lower the amount of times when a venue would be recognized as an event, either on Foursquare or on other channels. At first I though about an upgrade which would set the amount of people or tweets needed to trigger the "event discovered" action for a specific venue. This would enable us to reduce the importance of some venues, but it would also require manual work. Luckily, we came up with another brilliant idea: the more times a venue is trending, the harder it is for it to be trending again, at least for the next few days. Automatic balancing.

Venues with the most discovered events. Generic ones, besides massive places, such as train stations, cinemas, squares and shopping centers are too dominating.

The next problem: At this point, we have launched other test instances of Ljubljana Realtime (Maribor, Zagreb and Zurich), to see how the system behaves in other environments. Some cities are bigger, some are smaller, which means they produce different amount of activity. Besides, different services are used differently in different cultures.

Iteration 4: Supporting local instances

Foursquare is big in Croatia (Zagreb), but not so much in Switzerland (Zurich), which means Zagreb Realtime's stream had a lot of Foursquare trending posts, while Zurich's had a lot of "Increased activity on Twitter and Instagram" posts. It was obvious that local instances needed different algorithms. While having an option to set the amounts which would trigger the post on a specific venue would be too much to moderate, having the same logic on a specific region could work. And it does. Zagreb now needs more people checked-in on Foursquare, while Zurich needs more unique people tweeting or sharing photos.

Number of discovered events by type (Foursquare vs. Twitter + Instagram) on each day. Foursquare trending venues are dominating Zagreb, while social streams are dominating Zurich Realtime.

The next problem: The basic algorithm requires two different people to tweet/post from the same location in one hour. In case of Zurich, this amount was set to three, but it turns out this situation happens rarely, around 10 times fewer than with two people, or only two to three times a day. Obviously not enough.

Iteration 5: Improving the "increased activity" weight

You can only have a whole amount of people tweeting in the past hour. Two or three. In our case, we needed something in the range of 2 1/2. The modified solution adds the number of posts divided by ten to the number of users, which means that currently, at least two people making at least three posts in an hour will determine a trending event in Zurich. This is not a perfect solution from the event discovery view, but it does what urgently needed to be done: prevent having too many tweets in the stream.

The next problem: we currently have four Twitter accounts that tweet events for these four cities. Our target was for each of them to make around 10 - 15 tweets a day, which seems like a number that is not spam. But how can a person see which of these events is THE event?

Iteration 6: Going super venue level 2

The latest version of the algorithm now recognizes two levels of events. An event (mostly 6 people on Foursquare, mostly 2 different people tweeting), and an outstanding event (around 12 people on Foursquare, around 4 people tweeting). Our goal was to make this super event happen only once a few days, on rare occasions two times per day, and it has already happened a few times.

Sometimes super events happen, with tens of posts in a single hour, such as the one for Philips Fashion week. These events definitely require more exposure.

The next iterations

At this point, I'm very satisfied with how the algorithm works, even though a few other modifications need to be done (specially to support different days of week specifics and behavior). By measuring what is happening, learning from that information and building the next releases based on that knowledge, the activity stream logic has come a long way from the initial version. Measuring is crucial, and rarely we have went to such extent to enable this in the widest way possible (e.g. the update to balancing the posts based on the previous events would be trivial by itself, but we wanted to log things that would happen but didn't happen, besides things that actually happened).

These cycles of Build - Measure - Learn can be a lot hard work, but they provide great results, which are also very fun and rewarding. Some people simply need to see how deep the rabbit hole is. Do you have any other interesting cases or experience with this approach?

Discover what's happening in Ljubljana in real-time

Fri, 28 Sep 2012 13:37:29 GMT

I've always been a big fan of the power of the crowds. When a mass of people can achieve much more than a few skilled individuals can. And ever since we've started playing with Twitter's API, I've been think about the possibilities of this magnificent data source. Besides Twenity, we've done a few other Twitter mashups like Twitter walls, but this wasn't enough. We wanted something more - geolocation. Displaying information on a map in real-time. But since there aren't that many tweets equipped with GPS coordinates, we needed to include other services for more diversity as well. Which we did, and Ljubljana Realtime, a social event discovery application, was born.

Launch

Data and services

Ljubljana Realtime currently feeds on four different services: Twitter, Foursquare, Instagram and Flickr. It would be great if we could add other services as well, but Facebook doesn't allow public geo search, Google+ doesn't support geo search at all, and other services either aren't appropriate or don't offer an API which would allow us to get their data.

These four services are visited once a minute, and all posts in a radius of around 5km from Ljubljana city center are found: tweets, Foursquare trending venues, pictures from Instagram and Flickr. A variety of information created with different purposes on different occasions.

The area covered by Ljubljana Realtime. Different services require different searches, based on maximum allowed radius.

The application

These posts are displayed on a map, which was the original idea for the prototype. The last hours of posts on Google Maps, which can be zoomed and filtered at will. But the whole display felt a bit chaotic (still does), since there are many posts in vicinity of one another. That's why we knew we need to group similar posts, and we did this by the post's nearest Foursquare venue. Then a funny thing happened: this simple solution enabled something magnificent, something that could be much bigger than the whole posts-on-a-map application.

Anyways, since geo location is heavily connected with mobile devices, the application is responsive and fully compatible with most smartphones. Perhaps there will be native apps as well at one point.

The stream

Back to the magnificent. Originally, a Twitter bot was intended to come with the application (I love making those), which would tweet all trending foursquare venues to promote the application. But this seemed a bit lame, we needed to add something cooler. Something that would add more value and detect an event before a 4sq trending venue would happen. This is where the mentioned grouping of posts by venue came in handy, and the logic is as follows: if two or more people publish form the same venue in a single hour, this could very well mean something's happening there. And in most occasions, this turned out to be true. Read further for more details.

Discovering an event before a trending venue on Foursquare.

The event discovery stream is available on Twitter and Facebook.

The problems

Besides unstable APIs, the biggest problem we are currently facing is the geolocation itself. GPS chips in mobile phones are often not accurate enough, so people are located tens or hundreds of meters from their actual location. Combine that with the amount of Foursquare venues out there (imagine tall buildings), and you can understand Ljubljana Realtime sometimes has problems with connecting a post to a venue. Not to mention duplicated venues. We've eliminated some of this effect by only using venues with a certain amount of checkins and different users, but this will surely be the greatest challenge the project is facing in the future.

An event which is not.

The results

Problems aside, in most cases, Ljubljana Realtime event discovery works great. In a week or so since it's online, it discovered many events that were happening in Ljubljana (night run to the Castle, an athletic meeting, one of the first iPhones 5 in Slovenia, a public garage sale in park Tabor, etc.), and on many occasions, it discovered these events before a trending venue happened on Foursquare. Which is great. The potential is obviously there, and newer, improved versions and algorithms will surely behave even better.

A few of the great discoveries Ljubljana Realtime made.

The plans

This project is being developed in an agile way, where the application's behavior is constantly being monitored and changes deployed rapidly according to discovered strengths and weaknesses. The MVP (minimum viable product) is there, and with a few minor modifications, Ljubljana Realtime will soon be ready to expand to other cities and regions. Now it's up to you to help us, and it's pretty simple. When something magical is happening on and you are tweeting about it anyways, be a sport and click the arrow to include your geolocation in the tweet. By doing this, you will help others to discover what's going on in our beautiful city.

That's it for now, party on.

p.s. for all the Slovenians out there: the coordinates embedded in a tweet are pretty accurate even though Twitter will say you are in Italy. If you look at the picture of the map below the tweet, there's a polygon around Italy which sadly contains Slovenia as well. Hopefully, Twitter will remove bug someday.