Stritar's chronolog

Twitter beyond 140 characters

Sat, 23 Jan 2016 20:20:41 GMT

Twitter is in big trouble. The user growth has stopped and the world is waiting to see if Jack Dorsey can turn things around. One of the latest rumors is that Twitter will soon resort to one of its biggest gambles yet, and remove its signature 140-character limit, which has always been of the key differentiators of the service. Can this be true? Does this even make sense?

Let's start at the beginning. In my opinion, Twitter’s two main competitive advantages (compared to Facebook) are:

real time stream, which means relevancy. I’m not saying it's better than Facebook’s filtered stream, but it certainly delivers a different way of consuming content.
density of information, resulting from the above-mentioned limit, which means speed. This, again, makes a big difference in examining information, compared to similar services.

The combination of the two is what makes Twitter Twitter. At first glance, it seems increasing the tweet limit to 10,000 characters would totally mess up this user experience. But this doesn’t need to be the case, if the expansion is done properly. Actually, people have already been adding more characters to tweets by embedding print screens of quotes or articles as pictures. Why should additional tweet text be any different? The solution is simple, this upgrade has to be in the form of a text attachment.

A simple workaround of adding text as a picture that everybody's been using.

A link uses 23 characters of a tweet. A picture takes 24, the same goes for a poll or a quoted tweet. This logic could be applied to "extended tweets" as well, and as since I'm a big fan of Twitter, I'm truly hoping this is the way Jack will go. Information on the web always comes in multiple components. Google use titles and (meta) descriptions in search results, emails have a subject and a body, Facebook embedded links use a similar information structure, ditto for articles on news sites, etc. This actually makes sense. The microblog becomes the blog, if required, and everybody's happy.

A simple solution that solves a lot of problems and doesn't change the core user experience.

But if I’m wrong and actual tweets will support 10,000 characters in a single text field, Twitter is probably done for it. Removing this character limit feature will simply make the service too generic and not useful for many cases it is used for at this point. Lately, a lot of power users have already left Twitter, and if the rest of them leave as well, there won’t be any good reasons for new users to come in the first place.

UPDATE (23.2.2016): Awkward... It seems Twitter's Jack Dorsey has been talking about the same thing a few weeks before this post, which I somehow missed.

Solving global and local records in SaaS database design

Sat, 20 Jul 2013 23:50:56 GMT

Lately, we've been seriously considering developing a new version of our framework. This one is already a few years old, and besides other problems it's facing, it lacks one very important thing - it's not suited for SaaS (Software as a Service) applications. Often, we would like to host a few simple projects (like multiple web pages) in a single database, but we are also thinking about developing a product / service, which we could offer to multiple clients. Making a product for different clients that would live in the same database is not simple, and requires an architecture that is both rigid and flexible, micro-useful and scalable.

The database model we currently use is doing quite well. It's able to support complex business-oriented systems, social portals and web sites, the core tables being the same across all applications. Sure, there are many redundant things, but we are aware where we should scale-down. More importantly, we have to decide how to upgrade this database model which is used in different applications to be able to work in a single database. I'm not that fond of using multiple databases or schemas in SaaS architecture, since this is a similar situation to which we have now, and it doesn't work that well. I'm certain having the same database for different clients is most suited for our business.

This means we will be having a ClientID (ApplicationID) in all the tables that will be used by all clients / applications. But what is the best way to take care of global and local records - records that are used by everyone versus those in only a few applications?

Let's assume we are building a CMS system for many sites, for which we will be needing two global Roles (Administrator, Moderator), but one instance will have its own Role as well (News editor). There are many ways we can go.

1. Local roles

This architecture means each client having it's own Roles. It's a simple solution, very flexible, but has many flaws - the most obvious being the benefits of global functionalities. E.g. if you're and Administrator, you can restart the site. Since we now have multiple Administrators (ID = 1,3,…), features like these get a bit harder to implement, and you can end up developing a totally different application for each client.

ApplicationID	RoleID (PK)	Role
1	1	Administrator
1	2	Moderator
2	3	Administrator
2	4	Moderator
2	5	News editor

2. Global Roles with a M:N table

The other option is to have global Roles, which are attached to a single application using a M:N table. This is an elegant and very flexible solution, but from my experience, you should avoid using M:N tables if possible. Of course, there are logical cases in which you can't, but you should always consider other options. Adding another table to the equation complicates queries, makes direct data browsing less understandable, besides raising the possibilities of errors.

RoleID (PK)	Role
1	Administrator
2	Moderator
3	News editor

ApplicationID (PK)	RoleID (PK)
1	1
1	2
2	1
2	2
2	3

3. Composite key

Another possibility that comes to mind is to use a composite key, which in reality, makes a single table from the above two (Roles, ApplicationsRoles). This allows using the same IDs for the same Roles in different systems. But it also requires having a composite key, which makes other things more complicated. It prevents making simple joins (you need to make a join by both primary keys), which means all the tables joined to Roles would need the ApplicationID field as well, even if they wouldn't really require it. Not to mention this model simply calls for trouble.

ApplicationID (PK)	RoleID (PK)	Role
1	1	Administrator
1	2	Moderator
2	1	Administrator
2	2	Moderator
2	3	News editor

4. Fake composite key

Sometimes, it makes sense using an additional primary key in a M:N relationship, so you can easily join the table further. We could do this as well, making the previous case a bit more easy-going. But this situation makes all the magic that we could do across the system a bit harder - since all the joins would be made using new local IDs, we are getting too close to the first solution (you can't make joins with the RoleID field, since it's not unique).

ID (PK)	ApplicationID	RoleID	Role
1	1	1	Administrator
2	1	2	Moderator
3	2	1	Administrator
4	2	2	Moderator
5	2	3	News editor

5. NULL-able ApplicationID

A different architecture enables using global Roles that have no ApplicationID, and local Roles that have it. Each application is set to use both sets of Roles, allowing flexibility and understandability. But this model has a problem as well - we can't prevent an application from seeing a global Role - those are intended for everybody, which means that in the long run, the system would start gravitating towards not having any global roles at all, to assure flexibility. To simplify - this architecture is great, but needs something that takes care of Denying.

ApplicationID	RoleID (PK)	Role
NULL	1	Administrator
NULL	2	Moderator
2	3	News editor

5. a) Creating a table for Deny

Let's put another application (ApplicationID = 3) into the equation, which, for some reason, doesn't want to have the global Role named Moderator (RoleID = 2). Adding another table to care of Denied Roles would solve the problem perfectly, but do we really need another table? We want to keep our database with as little tables as possible, so it's easier to use. That's why we should consider extending the Roles table to handle denying as well.

ApplicationID	RoleID (PK)	Role
NULL	1	Administrator
NULL	2	Moderator
2	3	News editor

ApplicationID (PK)	DeniedRoleID (PK)
3	2

5. b) Adding two fields for Deny

In the most normalized way, we need two fields to take care of Deny - a boolean to mark we are working with a Deny record type, and an integer to tell which record we are denying. But what if we want to optimize even further?

ApplicationID	RoleID (PK)	Role	IsDenied	DeniedRoleID
NULL	1	Administrator	false	NULL
NULL	2	Moderator	false	NULL
2	3	News editor	false	NULL
3	4	NULL	true	2

5. c) Using an existing field for Deny

We could put information about which record we are denying into the existing Roles.Role field, since we don't really need it in these cases. But there's a major setback; if a non-numeric field would be accidentally stored in the denied Role, the SQL engine would throw an error.

ApplicationID	RoleID (PK)	Role	IsDenied
NULL	1	Administrator	false
NULL	2	Moderator	false
2	3	News editor	false
3	4	2	true

5. d) Using a single field for Deny

It's much easier to combine the first two fields into one, marking the record implicitly. If the DeniedID is NULL, it means we are working with a regular record, otherwise, we are dealing with a denying record. Information about which Role we are denying is stored in the same field, which works both as a boolean, as an integer.

ApplicationID	RoleID (PK)	Role	DeniedRoleID
NULL	1	Administrator	NULL
NULL	2	Moderator	NULL
2	3	News editor	NULL
3	4	NULL	2

The decision

The last architecture (5. d) is the one I think it's best. It's very transparent and flexible, and it has only one major problem I can currently think of; if only a few (of many) applications would require to share a Role, this wouldn't be possible to solve with this model, but I can live with that.

ApplicationID	RoleID (PK)	Role	DeniedRoleID
NULL	1	Administrator	NULL
NULL	2	Moderator	NULL
2	3	News editor	NULL
3	4	NULL	2

The SQL statement

I have played around a bit with the 5. d) and it works very well. Here's the SELECT statement that retrieves the records for each specific application, supporting the mentioned scenarios.

SELECT Roles.*
FROM Roles
WHERE
(Roles.ApplicationID = 3 OR Roles.ApplicationID IS NULL) --retrieve local and global roles
AND
Roles.DeniedRoleID IS NULL --use only regular records
AND
Roles.RoleID NOT IN --remove denied roles
(
SELECT DeniedRoles.DeniedRoleID FROM Roles AS DeniedRoles
WHERE DeniedRoles.ApplicationID = 3
AND DeniedRoles.DeniedRoleID IS NOT NULL
)

The same query, solved with a JOIN instead of WHERE IN, which is faster, but probably less understandable.

SELECT RealRoles.*
FROM Roles AS RealRoles --regular records
LEFT OUTER JOIN
Roles AS DeniedRoles --denying records
ON RealRoles.RoleID = DeniedRoles.DeniedRoleID --join by the ID of the role
AND RealRoles.DeniedRoleID IS NULL --but only those records that are real
AND DeniedRoles.DeniedRoleID IS NOT NULL --vs those that are denying
AND DeniedRoles.ApplicationID = 3
WHERE
(RealRoles.ApplicationID = 3 OR RealRoles.ApplicationID IS NULL) --retrieve my and global roles
AND
RealRoles.DeniedRoleID IS NULL -- take all regular roles
AND
DeniedRoles.RoleID IS NULL --that don't have a deny

Conclusion

There are many ways to solve a database model in a SaaS architecture, but because of many reasons, we've decided to use a single database with an additional foreign key that defines the client in all the required tables. One of the biggest concerns in this architecture is solving global and local records, but the model presented here takes care of most requirements a typical Saas system might stumble upon, by using a NULL-able ApplicationID, with another field to take care of exceptions that require denying.

This architecture solves most of the problems, and if you're leaning towards a single database architecture, it's probably the best way to go.

How the #hashtag took over the world

Wed, 22 May 2013 07:57:39 GMT

Ever since the times of IRC, there has been a need to categorize specific messages on the internet. With the rise of the social web and increased amounts of information, this habit became even stronger. First popularized by Twitter, the hashtag was introduced in 2007, and since then, found its way into standard offering of the most popular social services (Instagram, Tumblr, Google+, etc.). It seems even Facebook will introduce its own version soon, though people already use them overthere anyways. The hashtag changed the way we create and consume content, and it became a symbol of collaborative publishing in the social era.

On the technical side, the hashtag is nothing more than a clickable search query, a simple keyword that is attached to the post to add a bit of context to the content. An emoticon with a few more options. But for something so fundamentally basic, the hashtag turned out to be so much more, something almost magical. Sociology, communicology, computer science and other fields would probably have a lot to say about this phenomenon.

Today, hashtags form the ultimate crowdsourced media channels, some even managed to construct themselves as brands. It's hard to imagine something happening without its hashtag, and most TV shows and commercials already include them. Actually, they remind me of the Web's beginning, when everything had the AOL keyword attached to it, except now, keyword creation and curation is decentralized and distributed.

A graffiti in Ljubljana, Slovenia, persuading people to join the protests against corrupt elites. Hashtag included.

But there's something that's much more important than commercial applications: the planned or ad-hoc hashtags that form themselves in case of disasters or other extreme events. Millions can tune-in to to these instant channels, and participate with their own information. Such streams can go beyond a single service, existing above specific social media. These hashtags have the power to help people in need, and assist activists in making the world a better place. They are the ones that made the hashtag the word of the year 2012, marking the transfer of power from publishers to ordinary people.

How Facebook and Twitter are messing up Slovenian journalism

Tue, 08 Jan 2013 19:56:08 GMT

Before I proceed with this rant, I would like to point out that I strongly believe in social media. I think it can do magical things, hell, we've already seen it has the potential to change the world. But that's just me, a person consuming information, deciding what's real and what not, riding those waves that I like. The media - they should try harder - it's their job to report the truth. In the past few weeks, we've encountered a few slips from the mainstream media here in Slovenia, feeding us with bogus information they've copied from the (social) web. I know it can be hard to track everything that's going on, but still, double-checking a few things and sources wouldn't hurt that much. And since Slovenia is in a quite critical stage at this point, with a fucked up financial situation, austerity measures, protests and everything, this makes it even more important for journalists to do their job properly.

Case 1: the major mayor situation

Those that are acquainted with the current situation in Slovenia, know that we currently have two major protest venues. Maribor, who started it all, and whose protesters have already succeeded in persuading their mayor Franc Kangler to step down (and they aren't stopping there), and Ljubljana, whose activists are protesting more against the Slovenian right-winged government lead by Janez Janša, and no so much against the capital's mayor. You should know that Ljubljana's mayor Zoran Janković is an interesting person, one of those guys that's a bit shady, but since he has done so much for the development of the city, the general public still likes him (but not for much longer it seems).

In a similar fashion as the other protests (who are currently happening on a weekly basis), there was one protest in Ljubljana organized using Facebook and other social media channels, targeted specifically against the mayor. This event had hidden attendees, and their wall was covered with posts such as "10.000 confirmed guests", "15.000 confirmed guests" etc. There were a few of us that laughed at this, but this didn't stop the radio and television from reporting an expected record amount of protesters on that day, based solely on that Facebook event's wall posts! The result: a few hundred people, and some of them had a transparent "Thief, we love you".

The cached version of the event against Zoran Janković - with 15k hidden attendees. The event is no longer available. Spotted by @loudandwicked.

Case 2: the bogus response to the anti-corruption commission report

Today, another interesting thing has happened, and we still need to see how it will turn out. The national Commission for the Prevention of Corruption released a report saying both prime minister Janez Janša and Ljubljana's mayor Zoran Janković have more money that they were supposed to have (in hundred thousand euros). When this happened, Janša's SDS party's parody Twitter account with 30 followers at that time tweeted that this report proved the commission has finally dug into the corrupt communist hole.

The tweet that got mistaken for an official statement.

When I saw this tweet, I instantly checked the profile, since the tweet seemed very unbelievable, concerning the fact their leader is also inside the report. I instantly knew it's a bogus account, but this didn't prevent the Slovenian Press Agency from putting this tweet as a quote inside one of their news (no longer available), and from there on it was instantly picked up by other media as well. We are talking about one of the most important Slovenian news sources!

The citation of the tweet, copied by demokracija.si from STA. Spotted by @miharejc, amplified by @multikultivator, read the the thread.

Time for real journalists to shine

There are probably more cases like this, and it needs to stop! Social media is a wonderful tool for providing additional information about something specific, but as we've seen in the two cases, it often can't be the primary source for the news. Journalism needs to go a step back, and provide credible information from trustworthy sources. Otherwise, there won't be much truth left for our descendants, since too many people will simply have the power to make things up. But everything's not bad - situations like this could provide clear distinction between real journalists with original sources, and those who do nothing else but copy others. Something we critically need in this age of data abundance.

Using JSON (with asp.net) is like wiping your ass with silk

Sat, 05 Jan 2013 18:45:44 GMT

I love mashups. Actually, I love everything about them, I love using them, I love making them, I love those who do everything they can to empower them. In my opinion, mashups are one of the most significant concepts the Web has invented, since they represent unlimited possibilities of integrating and reshaping things that are already done. The platforms out there are stable, so it's the creativity that sets the limits. These days, you can easily take data from anyone and do something else with it. Just don't forget to use JSON.

I'm an old school guy who has been working mostly in enterprise software environments, which means using XML for data integrations, besides, RSS used to be the man. The first generation of Twenity (Twitfluence) was using XML feeds, and this blog is using mostly RSS for fetching entries from other sources. XML was the standard we all spoke, very cute and readable, but on the other hand, not so easy to parse. At that time, this fact didn't represent such a major problem, since a few more lines of code took care of everything. But today is a different situation.

It was Twitter who first started dropping support for XML, which annoyed the hell out of me. I had to start rewriting things for JSON. This turned out to be one of the best things that ever happened, since JSON seems to be loved by everyone. It's super easy to use with jQuery, but since I'm a server-side type of a guy, it's even more important that it's cleverly integrated into asp.net. Fetch the data, store it, do crazy shit with it.

JSON is serializable into a .net object with a single line of code. Create the class with parameters compliant with the the specific JSON structure, serialize the response string into that class, and everything automagically works. Piece of cake, unlimited opportunities. Take Twitter for example:

Make the basic Tweet class:

public class Tweet
{
public string id_str;
public string text;
}

Create the request to access a tweet (funny, the hardest thing to do):

string url = "http://api.twitter.com/1/statuses/show/274508827146215424.json";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = "GET";
WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string jsonResponse = reader.ReadToEnd();
reader.Close();

Which will return something like this (play here):

{
"created_at": "Fri Nov 30 13:42:59 +0000 2012",
"id_str": "274508827146215424",
"text": ""Facebook knows what we say, Google knows what we think",
"source": "web",
...
}

Serialize the response string into an object:

JavaScriptSerializer js = new JavaScriptSerializer();
Tweet tweet = new Tweet();
tweet = js.Deserialize<Tweet>(jsonResponse);
Response.Write(tweet.text);

Pretty neat. Since I've started using JSON, mashups have become easier than ever to make. With one of our latest projects, we've integrated our application with Twitter, Instagram, Foursquare and Flickr. For breakfast! XML may have its advantages, but for such things, JSON is simply the greatest. All hail the new lord!

A case study in agile development: the algorithm for Ljubljana Realtime's event discovery

Mon, 22 Oct 2012 20:01:16 GMT

When we started working on Ljubljana Realtime, we decided to approach it in an agile way. Amongst others, we wanted to use a few interesting lean concepts such as rapid development, Minimum Viable Product and the Build - Measure - Learn iterations. Less than two months later, the results are in, and they are very pleasing. The MVP in the shape of an activity map was developed in a few weeks, only to show there is a lot of social noise which will somehow need to be taken under control. But that's currently low priority, since the first pivot is already taking place, slowly shifting the focus from the rich map application towards an event discovery algorithm and stream. That's where I see the most potential of Ljubljana Realtime, and in the last weeks, that's where the most work was done.

Launch

Ljubljana Realtime event discovery engine uses Foursquare trending venues and geo-tagged posts from Twitter, Instagram and Flickr to discover what's happening in real life. At least 6 people checked-in on Foursquare or two different people tweeting or posting photos in a single hour could mean something is going on. These events are posted to Twitter and Facebook, with links to the posts. A few versions of this algorithm were already deployed, each one solving new problems, resulting in a few micro Build - Measure - Learn cycles in a single month.

Iteration 1: Foursquare, no duplicates

The first version of the stream (bot) was a simple one, at that point it was meant to work as promotion for the map. The only thing it knew how to do was to wait a few hours until it posted the same thing again. I think Foursquare checkins are alive for three hours, so if a trending venue was still trending after that time, new people had to checkin and the venue was still buzzing.

Problem: Plain, no real added value.

Iteration 2: Adding activity from other sources

When we were trying to make some space on the crowded map, we started grouping posts from Twitter and Instagram by the nearest Foursquare venue, which meant having less boxes on the screen. This turned out to be quite a complex thing to do properly, but it was worth the effort. On only a few occasions, one venue would have multiple posts in a single hour, and in most cases, that meant something was happening there. This provided another very interesting potential for the activity stream. Actually, it made the stream bigger than the map could ever be.

(I love it when such things happen, when you are trying to solve a problem, and it turns out there is much more hidden behind the resolution.)

Groupping posts by a venue. Did Ljubljana Realtime just discover an athletic meeting taking place?

The next problem: Activity in some venues, specially generic ones such as "Ljubljana" would trigger the stream almost every day. Similarly, some large venues, such as supermarkets, would be trending too many times on Foursquare.

Iteration 3: Balancing the posts

The algorithm needed an update, which would lower the amount of times when a venue would be recognized as an event, either on Foursquare or on other channels. At first I though about an upgrade which would set the amount of people or tweets needed to trigger the "event discovered" action for a specific venue. This would enable us to reduce the importance of some venues, but it would also require manual work. Luckily, we came up with another brilliant idea: the more times a venue is trending, the harder it is for it to be trending again, at least for the next few days. Automatic balancing.

Venues with the most discovered events. Generic ones, besides massive places, such as train stations, cinemas, squares and shopping centers are too dominating.

The next problem: At this point, we have launched other test instances of Ljubljana Realtime (Maribor, Zagreb and Zurich), to see how the system behaves in other environments. Some cities are bigger, some are smaller, which means they produce different amount of activity. Besides, different services are used differently in different cultures.

Iteration 4: Supporting local instances

Foursquare is big in Croatia (Zagreb), but not so much in Switzerland (Zurich), which means Zagreb Realtime's stream had a lot of Foursquare trending posts, while Zurich's had a lot of "Increased activity on Twitter and Instagram" posts. It was obvious that local instances needed different algorithms. While having an option to set the amounts which would trigger the post on a specific venue would be too much to moderate, having the same logic on a specific region could work. And it does. Zagreb now needs more people checked-in on Foursquare, while Zurich needs more unique people tweeting or sharing photos.

Number of discovered events by type (Foursquare vs. Twitter + Instagram) on each day. Foursquare trending venues are dominating Zagreb, while social streams are dominating Zurich Realtime.

The next problem: The basic algorithm requires two different people to tweet/post from the same location in one hour. In case of Zurich, this amount was set to three, but it turns out this situation happens rarely, around 10 times fewer than with two people, or only two to three times a day. Obviously not enough.

Iteration 5: Improving the "increased activity" weight

You can only have a whole amount of people tweeting in the past hour. Two or three. In our case, we needed something in the range of 2 1/2. The modified solution adds the number of posts divided by ten to the number of users, which means that currently, at least two people making at least three posts in an hour will determine a trending event in Zurich. This is not a perfect solution from the event discovery view, but it does what urgently needed to be done: prevent having too many tweets in the stream.

The next problem: we currently have four Twitter accounts that tweet events for these four cities. Our target was for each of them to make around 10 - 15 tweets a day, which seems like a number that is not spam. But how can a person see which of these events is THE event?

Iteration 6: Going super venue level 2

The latest version of the algorithm now recognizes two levels of events. An event (mostly 6 people on Foursquare, mostly 2 different people tweeting), and an outstanding event (around 12 people on Foursquare, around 4 people tweeting). Our goal was to make this super event happen only once a few days, on rare occasions two times per day, and it has already happened a few times.

Sometimes super events happen, with tens of posts in a single hour, such as the one for Philips Fashion week. These events definitely require more exposure.

The next iterations

At this point, I'm very satisfied with how the algorithm works, even though a few other modifications need to be done (specially to support different days of week specifics and behavior). By measuring what is happening, learning from that information and building the next releases based on that knowledge, the activity stream logic has come a long way from the initial version. Measuring is crucial, and rarely we have went to such extent to enable this in the widest way possible (e.g. the update to balancing the posts based on the previous events would be trivial by itself, but we wanted to log things that would happen but didn't happen, besides things that actually happened).

These cycles of Build - Measure - Learn can be a lot hard work, but they provide great results, which are also very fun and rewarding. Some people simply need to see how deep the rabbit hole is. Do you have any other interesting cases or experience with this approach?

Dear Twitter, please reconsider this madness

Tue, 21 Aug 2012 15:01:53 GMT

Dear Twitter. You were always one of my favorites. I love it how you managed to create a subculture, something that Facebook will never be able to do. You've allowed people to collectively create information that travels the world in seconds. Your role in many global activist projects is invaluable, and your stance on privacy issues something that inspires the world. You help us find out about things before they actually happen. When the Icelandic volcano was erupting in 2010, I was watching the #ashtag word clouds being broadcasted live on CNN. It was then that it struck me that the media will never be the same again, the power of thousands reporters wins every time. You made it to the mainstream, shaped transmedia into Twitter + television. You were becoming the ultimate foundation for real-time information. You made me realize so many things by being one of the most prominent platforms of the social era. You were the man.

And now you are starting to kill the platform.

Anyone who's read What Would Google Do? would know this is a huge mistake. This book (from 2009!) has a chapter about the value of platforms and distributed systems, and I've been obsessing myself with this revelation ever since. Wrote a blog post about it, which was retweeted by the book's author Jeff Jarvis himself, and it made me even more in love with the idea. Now I'm sure platforms are the future, because they have the ability to outlast services.

The most important things about platforms are the ecosystems around them. Yours was always more appealing to me than Facebook's. Because of the information Twitter has (almost all of it public), because of the culture that's emerged around this information. Twitterers, Developers, even Lurkers and Ninjas, the amount of innovative things that people have managed to build around your core is amazing. Not just silly games and marketing schemes, actually valuable and inspiring things (we've tried it too!). I always believed you were cooler than Facebook, that you have a brighter future, since you weren't trying to keep people inside your services. You seemed to know what made you in the first place.

I understand it's the high level mashups you are interested in, but still, the platform is the most important thing you have. Why not let other people reinvent your user experience, like others are trying to reinvent email? You want that, you want people to make Twitter whatever they want, while creating, consuming and curating the data that's inside you. You need thousand of different clients that feed your infinite hunger, enabling you to build your business model around the tweets that are generated. You want developers to use, abuse and reshape your essence. Because platforms are like networks, there value grows exponentially with the number of nodes they have, and these nodes help platforms evolve even further, in unimaginable ways.

And now you are trying to kill bits of this platform, those bits that help create the most valuable thing you have - hundreds of millions of Tweets every day. They may be just Twitter clients, but they might as well be the most important thing you've ever had. Your fans, your supporters, your ambassadors, your army, your ecosystem. Your future is more important than your monetization, not just to me and to them, to everybody who loves what you are, and to those who don't get you. Your future is important to mankind. That's why you should stay what you were meant to be. You should stay the open platform.

Is it even possible to create original content in this age?

Sun, 15 Jul 2012 11:52:13 GMT

I've been blogging for more than three years now. And I love doing it, hell, I think my blog is pretty fucking awesome. But there's a problem I've been noticing lately. Every single piece of content I write has probably been written hundred times before. By mainstream media, by authors, by bloggers, by you. There's no way around it, and it bugs me to infinity. Take this specific post for instance, I won't even google it, but I can guess plenty of others writers have faced these thoughts and wrote about them from their own viewpoints. The irony of the situation is fantastic.

The amount of information on the internet is overwhelming. Thousands of blogposts created every day, hours of video uploaded to YouTube every minute, infinite tweets (check out this cool infographic). And only so many things you can tell. Which pretty much means that no matter what you write, someone else surely wrote it before you. All the topics have been covered so many times it's silly. You feel you're stealing other people's ideas even if you aren't. Or are you?

I'm one of those people who tries to absorb as much information as possible. I want to learn every day, synthesizing the gathered knowledge for various reasons, such as my work or this blog. From this perspective, I'm inevitably copying other people's ideas, rewriting content that's already been written many times before. Stealing Lego blocks and using them to build new sculptures. Is there even such a thing as an original idea? There probably really aren't that many.

Writers and bloggers are just a bunch of monkeys who are trying to write Shakespeare anyways. Curators, who are striving to separate the sheep from the goats, blending extraordinary ideas into somethings even greater. And that's what matters, that's where the value is added. Even though most of the content is not original, if any. But I guess that's ok, I just needed to think and write about it, which helped me understand this is the very essence of blogging.

Do you think we are slowly reaching the end (of science)?

Sat, 07 Jul 2012 09:59:15 GMT

This was great week for science. The scientists from CERN Large Hadron Collider finally proved with great probability that the Higgs boson particle exists. Not that any of us mortals truly understand what it means for the future of mankind, but it's supposed to be quite significant, so I won't argue with that. Science has come a long way, and while we take into account a few other interesting and revolutionary fields, such as Artificial Intelligence, Biotechnolooy, Nuclear Fission, Stem Cells, Genetics, etc., we must also consider the timeframe in which these discoveries did or will take place, in relation to the history of our planet and humanity.

Imagine this: if history of Earth would be represented by one standard year, first cells would appear by the end of February, life would move to dry land around December 1st, dinosaurs would appear on December 13th, modern mammals on December 27th. On the evening on December 31st, first hominids would evolve in Africa, and 10 minutes before midnight, Neanderthals would spread throughout Europe. Around minute to midnight, agriculture would be invented, the Roman Empire would collapse 10 seconds before new year's, and the last 2 seconds would be marked by the industrial era. (source)

Makes you think about our insignificance in the history of Earth (not even the whole universe). But we've come a long way in these few minutes. The steam engine was invented 250 years ago, and look at us now. All in two seconds of Earth's history!

I think it was Isaac Asimov who said modern computerization represents the fourth big revolution in communication, after speech, writing and printing. Speech was invented tens of thousands of years ago, writing thousands of years ago, printing hundreds of years ago, and information technologies decades ago. Yes, it goes exponentially faster with each iteration, because every new generation of communication allows information to travel faster and reach more people, besides the fact that each time, global population is a few times greater than before. This enables science and knowledge to evolve even on a faster pace. Thousands of universities and scientist are already taking into account what CERN had discovered and announced, developing theories even further, making experiments of their own.

If all goes well, I think there is only one possible scenario: we will soon come to the end. Find that particle, understand that impulse, define that force. And by soon I don't mean tomorrow, or even in our generation, but in no time compared to History. Hundred, even thousand years? Why not. Ten, hundred thousand years? Sure, we have time, what are a few "minutes" more. Imagine everything we'll discover if we don't fuck it up on a major scale and our society evolves in a similar fashion as it did before.

But what is waiting for us at the end? Well, I think we are all pretty much aware of that. "Why are we here?", "What do we have to do?", "Is this real life?" and that sort of shit that brought us here in the first place. I think science has a good chance of clearing these things up one day, and then we will find the ultimate answer (or question). And Science will become one with Religion and the whole situation will be quite ironic, concerning the traditionally opposed stance they have. The only question that remains is, will we live happily ever after then, or will we just find another, greater and even more complex system, where we will have to help our Gods find their Gods?

The final destination, part 1: technologies and concepts enterprise IT will have to adopt

Mon, 02 Jul 2012 18:38:09 GMT

In the past years, we've witnessed a very important transformation: the consumerization of information technologies. Billions of connected users living their life online, overwhelmed by millions of information systems that have been tailored to suit their every need and desire. Google, Facebook, Apple and Amazon came a long way with their products and infrastructure, but the enterprise isn't losing any time. Learning from the new paradigms and adopting new funky technologies, that have traditionally been developed in corporate laboratories. Can the Fab 4 also predict where enterprise IT is headed? And what will it become?

A few weeks ago I participated in a panel for PI-PL on Ljubljana's Faculty of Economics, where I was asked this exact question: where do I see corporate IT in 10 years. A very hard question indeed, but the more I thought about the it, the clearer it became. Enterprise data, software and technology will sooner or later integrate everything. Simple as that. But to fully understand how this will happen, we must first try to identify the most important trends that have shaped information technologies as we know them today. Yes, most of them don't have that much to do with the enterprise. But things are changing.

Cloud technologies

While cloud computing may have been present in the enterprise for quite some time, it's still pretty much dominated by web players like Google (mail, docs, etc.) and Amazon (hardware), who are also flirting with the enterprise. Who wouldn't? There are massive benefits for businesses to move their stuff to the cloud, from scalable physical Infrastructure to higher level Platform or Software as a service information systems.

Perhaps the most important thing the Cloud achieved was to render technology infrastructure irrelevant. It doesn't matter any more, what kind of environment you use. What type of security, infrastructure, servers and network you have installed. You can outsource these things to others, and it will be much easier and cheaper, while all your migrating-to-a-bigger-thing problems will be solved with a swipe of a credit card. I was fascinated that Microsoft now offers Linux based servers on their Azure cloud services, which can be changed to Windows with a click of a button. Architecture doesn't matter anymore, and this fact helps IT departments to focus on more important things than system administration.

APIs, mashups, platforms and ecosystems

Heavily connected with the whole Cloud concept, data and information never had it easier to travel from one place to another. System migrations (moving data from one information system to another) and system integrations (connecting multiple information systems into one) have always been one of the biggest challenges of IT. But the web didn't have as much resources as the enterprise, so it had to simplify things. By offering APIs (Application Programming Interface), web applications allowed others applications to work with their data in an easy way. Mashups, hybrid information systems built on top of others, were born.

Pretty much every noteworthy web service has its own API. This helped a lot of them to become a platform. You know, like Facebook and Twitter, who have created an ecosystem, where thousands of other applications live around them? Soon, similar concepts will dominate the enterprise too. There are already players like Salesforce, who not only offer business-oriented Software as a service solutions, but the also a Platform for other developers to build services on top of their services. And since everything is so open, all this data can easily be integrated with other information systems or transferred to a different environment. Modern information systems don't have problems with understanding each other, but IT departments have problems with understanding information systems, since different, more business oriented skills are needed to support these integrations.

Mobile devices and new distribution channels

More than ten years ago, when I was an Information Sciences student, there was still a debate going on about the benefits of web based enterprise information systems over traditional Client - Server architecture. In the end, the Web won, mostly because distribution was so easy, you make the update on the server, and every user gets it instantly. Employees need nothing but a browser. They are acquainted with the environment ever since they started using Hotmail, and took it for their own ever since they started using Facebook.

But today, it seems the Web is losing its ground as the leading infrastructure, since a new technology came into town. Capable mobile devices, like smartphones and tablets, now enable access to information systems from anywhere, anytime in real-time. Besides, they arrived with their own app markets, which enabled a whole potential for software distribution, and perhaps more importantly, for software billing. You give a fair share to the store owner, who also promotes your solution, and you can freely focus on developing and marketing the product. It's true that mobile apps may not be as flexible as web applications, since the users need to install the updates (even though this can also be achieved by combining native mobile and hosted HTML 5), but the trend is clear. Apple already has its Mac store, Microsoft Windows will follow soon. Distribution of mobile and Software as a service information systems is becoming trivial.

Big data and The internet of things

Traditionally, banks, retailers and financial institutions have been the organizations that operated with the most data in the world. Well, things are changing, and we can only wonder who owns the most bytes today: is it Google, Facebook or someone else? Since there are less transactions than there are interactions, we can estimate consumer oriented information systems with billions of users are the biggest in existence. While this data is accessible to the enterprise to some extent, there are also hundreds of other systems the enterprise or its employees use, and they all create massive amounts of data and information, which needs to be integrated into a wider picture.

Not only that. Today, there are already are more connected devices that we have initially anticipated. These devices (cameras, sensors, tools, etc.) create even more data, which the enterprise needs to process. This trend of wired gadgets is called The internet of things, and together with the large amount of interconnectable information systems businesses use, points to one important trend: the typical enterprise was never faced with so much data and information, which somehow needs to be integrated and understood in an interdisciplinary way.

New ways of doing things, on a higher level

In the mean time, these consumer oriented (B2C) web corporation not only became a few of the biggest technology companies in existence, they've also invented new ways of how to get things done. From Google's 20%, to flat organizations without management, more and more companies (not only startups) set out to revolutionize how business is done. In the service oriented society, creativity is important, but so is productivity and the ability to ship fast. Done is better than perfect.

With new types of management concepts, such as lean and agile, modern organizations are becoming more and more flexible. Not only in doing things, but also in switching from one technology to another. These companies have developed their own way of thinking about which software to use. And it probably has a lot to do its price, how fast can you start using it, how scalable and connectable it is, and how fast can you dump if for another. IT requirements are moving to a higher level, and information systems have become just pieces of a puzzle that needs to be completed.

Design and user experience

Design and user experience probably still don't have that much to do with enterprise IT, but they are very much worth mentioning nevertheless. Face it, users are becoming more and more demanding, and software developers need to make better and better software. Even though the above mentioned facts are probably the dominating factor for the choice of which information systems the enterprise will use, design and user experience matter more and more.

Businesses have always had problems with implementing new software, educating the users, going through the whole status quo change. But beautiful and useful software penetrates faster. People perceive beautiful things to be more useful, and it's the whole intuitiveness and usability of software that helps them adopt something without too much resistance and problems. Some software vendors already found out user experience is the new competitive advantage, and in the end it may be the thing that tips the scale. But the whole point behind it is that I can see better, more clever and detailed, information systems force out older ones on an even faster pace. The whole world of information systems need reinvention, and user experience design will be the science behind these upgrades. Benefits are becoming more important than features.

Social, crowdsourcing and gamification

More than 3 years ago, I was very excited to present a concept we have been developing in Neolab on Days of Slovenian IT. I called it IT 2.0, since it meant integrating social services into enterprise software (at that time, Enterprise 2.0 was more widely used for stand-alone social software such as wikis or corporate blogs). The truth is, I didn't get the chance to sell it well, and in the mean time, disruptive social services managed to fully find their way into the enterprise. But business won't stop here; there are many other fascinating things the internet has invented that can fully be applied to corporate environments.

Gamification, the art of using gaming mechanics in non gaming environments, is getting more and more claim beyond the web. Both for motivating employees, as for motivating clients. We all like to play, so why should work be any different? And we all like to participate in something bigger, that is why crowdsourcing, where people coproduce something, can bring such exciting results. Can you see where I'm headed?

We are all human, and in the end, behaving on a very basic level. Sometimes we perform better, sometimes worse, and we all hold hidden potential even ourselves aren't aware of. That is what software in the workplace can sometimes help us discover, and it's something more and more businesses are aware of. Social, gamification and crowdsourcing are only a few approaches that can make us feel better and more motivated, and they are all concepts that enterprise IT will adopt sooner or later. Behind every company, there are only people.

The final destination of enterprise IT

These are the concepts and technologies that will shape enterprise IT of tomorrow. And with it, new challenges will emerge, together with new profiles of people, who will understand and use all of the above. These profiles, such as data scientists and business analysts, will help enterprise IT do what it was destined to do: Integrate life, the universe and everything.

But that's another story. Coming up soon.

Check out the complete The final destination series.

The future (of software) is in platforms

Sun, 08 Apr 2012 16:26:11 GMT

I don't know if you've had the chance to read What Would Google Do? by Jeff Jarvis. You should, it's a very powerful book, even though it's been written a few years ago. Things have changed a bit since then, when Google was on top of it's game, but that doesn't mean the ideas presented in the book aren't more actual than ever. One of the chapters that made the biggest impact on me was the one about platforms and distributed systems. Google managed to conquer the world of Web 1.0 by being decentralized, allowing others to embed YouTube videos, Google Maps and Ads anywhere on the Web. This orientation provided the fuel for Google's further development and growth. Today, this way of thinking is not a competitive advantage anymore, it's becoming a necessity. As you will see, current online market leaders of various industries are not those who provide the service, they're the ones who provide the platform.

Platforms rule because mashups rule

Platforms, as opposed to products and services, allow others to build on top of them. Not thinking about control and centralization, they provide the basic building blocks other can use to develop even more products and services. In the world of Web 1.0, this meant using an embedded YouTube video instead of having your own video player, but Web 2.0 has been heavily defined by ecosystems of services built around other services. With APIs these platforms provide, developing high-level mashups has never been easier from the technical point of view, and this type of architecture benefits everybody. The platform vendor gets additional developers that extend and market it's service, while the satellite mashup gets the distribution channels, users and data they need to get somewhere faster. A modern synergy packed symbiosis.

Oren Michels, CEO of Mashery, which provides API as a service, talking for Silicon Gardens in Kiberpipa

The technology platforms

Today, pretty much all of the technology market leaders are platforms. Google allows various levels of use of it's services, from embedding things to using different APIs. Facebook's games and other apps ecosystem is huge, with providers such as Zynga already gone public. Twitter and it's massive amounts of tweets generated in real-time produced the Twitterverse - an array of fascinating high-level services. Foursquare, a geo-location network allowed other interesting concepts, such as augmented reality and games, on top of its service. Amazon offers you to build your own store, and Apple's and Android's mobile app stores are hosting hundreds of thousands of apps developers can build, deploy and distribute in short amounts of time. Salesforce offers a platform for enterprise IT, WordPress for writing and blogging tools. I could go on, but you get the picture. Platforms are the future, because they evolve collaboratively, with thousands, if not millions of people co-creating them.

A massive Salesforce conference in San Francisco, which I've visited on my Silicon Valley trip (thanks Davorin!)

The real-life platforms

Not only in software and technology, platforms in real-life are also becoming more widespread. The whole Apple iPad/iPhone gadgets ecosystem is one of the most obvious cases, where various providers offer sound systems, dongles, add-ons and other accessories that upgrade and make the original item even more appealing. Nespresso and Dolce Gusto are another interesting products. Nestlé provides the platform - the small coffee pads, which they are pushing to the market, while different manufacturers make coffee machines and other complementary products. The platform becomes more useful because of its satellites, which make the platform even more successful, turning this relationship into a virtuous circle.

Coffee machines by various vendors for Nestlé Dolce Gusto

So what should I do?

Firstly, you should embrace this way of thinking and understand what's happening. Platforms have been around since the beginning of software, even though you maybe haven't thought about them in such a way. Computer architectures, Operating systems, Programming languages and Development environments are all platforms, on a more basic level - but they provide the foundation which others can build on top of. Other platforms are build atop of these and others atop of those. Platforms are everywhere, and with every new level, they are less technically, and more conceptually oriented. Think about where you fit in.

Next, find a platform that suits what you are trying to do. Personally, against many odds, I believe more in Twitter than I do in Facebook, since Facebook is trying to keep as much as possible inside its service, while Twitter acts distributed. One of our projects is Twenity, a game we've developed around Twitter influence measuring.

If you're feeling lucky, you might want to become a platform on your own. While this may be almost impossible to do, it's something I wish I will have the chance to do someday. Neolab already has its own development framework, and if we do manage to make enough surplus or find an investor that will take us down this road, I'll be the first one in line. That is, if we don't decide to join an established platform and rather build on top of that.

Things have changed. The future (of software) is in platforms. Are you already a part of the show?

The Silicon Valley tour, part 6: A trip to the Internet Archive

Wed, 21 Mar 2012 18:28:09 GMT

Those that have read my previous post about visiting the technology giants of Silicon Valley, might have gotten the idea that organizations around here aren't that welcoming to strangers. Well, that might not be entirely true. One of my stops in San Francisco also included a visit to the Internet Archive, a foundation that is trying to preserve all the information our civilization possesses. And they were more than welcoming. Besides giving Andraz and me a full tour of their headquarters, they've also invited us to one of their staff meetings, where the Archive's members and volunteers present their activities and results from their specific fields.

The Internet Archive is located in a former church in San Francisco, so there's more than enough room for all the people and hardware. Two main projects are currently taking place; one is the Wayback Machine, which makes snapshots of websites throughout history, and the other is the Archive, which represents systematic digitalization of movies, music and books, manually done by its members. A never-ending task that can only be done by activists who genuinely want to protect the knowledge of mankind, and you can contribute to the effort as well.

During our tour, 2.5 petabytes (that's 2.5 million gigabytes!) of data was mentioned, but I'm not sure if that's just one of the projects or both. This data is well protected, with mirrors all over the world. Impressive stuff, good enough to currently make archive.org the 223rd most visited site in the world (according to Alexa), get to the front page of The New York Times, and receive various prestigious awards.

One of the most fascinating stops of my Silicon Valley trip was indeed educational and inspiring, and it got me thinking: If there's a modern Library of Alexandria in the making, this would be it. Thumbs up.

Internet Archive's Headquarters in San Franscisco. It goes well with the logo, even though they came here only a few years ago.

The staff meeting, where members from various fields presented their achievements

Everyone that is an employee for more than 3 years, gets its own figure inside the church

Brewster Kahle, the chairman, passionately presenting the Archive's servers. There is no air condition, since heat is used for building heating.

The irony: since consumer hard disks are cheaper, they buy those and get rid of the casings.

This is how scanning of videos, microfilms and books looks like

Are we there yet?

Trademarks and logos are the property of their respective owners.

Check out the complete The Silicon Valley tour series.

The Silicon Valley tour, part 3: The magnificent Stanford University

Fri, 09 Mar 2012 03:21:33 GMT

There probably aren't many institutions associated with Silicon Valley the way Stanford University is. Its affiliates and graduates played a major role in the development of the southern part of the San Francisco Bay Area, which would later on become known as the Silicon Valley. The spirit of entrepreneurship, technology, science and research is felt everywhere, and Stanford University will surely be one of the most fascinating stops on my Silicon Valley trip.

Stanford University can only be described as the American educational dream. At least the way Europeans imagine it, thanks of pop culture and television shows such as Beverly Hills 90210. The perfect mixture of easiness and seriousness is manifested in the campus itself, which is filled with amazing parks and glorious squares and buildings. It's beautiful, but also tempting for a person like me, who is still playing around with the idea of being a student for a few more years.

We managed to spend some time with Jure Leskovec from Slovenia, who is an assistant professor there. His research is focused mostly on data mining and network analysis, very fascinating fields that I'm very interested in too. He gave as a tour of the campus and took us to the Institute of Design, an inspiring place where all sort of crazy things are researched and developed. Later on he proudly presented his lab, which was sponsored by Google. A great-looking place indeed, I hope he didn't notice me drooling all over it.

Stanford University - the mother of all knowledge, located right in the heart of Silicon Valley. Seeing it helps you understand why this region is miles ahead of anyone else in technology, and definitely makes you want to study there. It's simply magnificent.

The Oval: a beautiful park, located in front of the main entrance from Palo Alto

The view on the campus from the Hoover tower

One of the beautiful streets

Arches are practically everywhere

The Institute of design: some students programming robots

The Institute of design: results from a few intense brainstorming sessions

Jure Leskovec's lab, where he and his students do their magic

Check out the complete The Silicon Valley tour series.

Can social content curation without negative actions (dislikes, downvotes) even exist?

Thu, 19 Jan 2012 17:04:56 GMT

There are people who create content. Millions of them, producing enormous amounts of data and information every day. On the opposite side, you have the consumers, people who absorb most of this content for various reasons. And there are those in-between, an emerging layer of people who filter this content and pass the one worth consuming forward to others. These people are called content curators, a breed that's becoming more and more important these days, perhaps even more important than the original creators themselves. After all, they're the ones categorizing and cleaning up the chaotic Web.

Together with the dawn of the interest graph and information overload, it's becoming obvious content curation is a major trend in social. Mathematical aggregators and algorithms can only do so much, but they are no match against the collective effort of millions of people. This fact helped services supporting social curation to emerge everywhere, in different shapes. Twitter, Reddit, Digg, Delicious, StumbleUpon, Tumblr, Pinterest, even Facebook and Google search are offering you a filtered experience of the Web, an experience that is determined more or less by other people who share your interests. Social curation at its best.

I love these services, but as I've become a heavy user, I also started wondering, if they can work on the long run using only positive actions. Likes, upvotes, diggs, +1s, retweets, reblogs. All used to amplify, not suppress. But what if I want to prevent something from spreading, wouldn't that be curation too? Wouldn't that be a statement that would suggest the creator / curator to try harder? Another problem: how can you prevent an organized group of people (or people with multiple accounts) who are all trying to push something forward, from ruining the experience for others, without the ability to quiet them down?

I remember when Digg and Reddit were in the same league. They both used upvotes and downvotes and had similar traffic and the number of upvotes on the front page. A few years later, Digg was experimenting with becoming more Twitter-like, canceling the downvotes and introducing following. They failed miserably, but also because Digg front page was supposedly controlled by the power users. The ugly side-effect of content curation without proper goals and (crowdsourced) control. But things like that happen, and there's a good reason we probably won't see the Facebook Dislike button anytime soon.

Curation services, constructed around recommendation (Digg, Reddit, StumbleUpon), recognized the need for negative action and allow (anonymous) dislikes. On the other hand, those services, that are built more around social (Facebook, Twitter, Google+), can't afford to have them without causing a social problem, creating Enemies from Friends.

But there's always an option. An option that is sadly way more drastic than a dislike could ever be. Unfollow. Unfriend. Unsubscribe. Report. Remove yourself from something. An action that not only discontinues the information flow, but also discredits the creator / curator in a way. And it may as well be the ultimate dislike. Like it or unlike it, content curation without the negative action can't exist.

Did Google just admit Apple's Siri is the future of search?

Sun, 04 Dec 2011 15:21:19 GMT

I don't know if you saw The evolution of Google search video, which they've published a few days ago. You should, it's a cool movie, portraying the history of search and Google's vision of its future. But something went wrong. One of the punchlines of the video was a story from one of the engineers, who said that next-generation search engines will be able to answer complex questions such as the following:

"Hey, what is the best time for me to sow seeds in India given that monsoon was early this year?’"

A very legitimate question.

I don't know if you've tried out iPhone's new personal assistant, Siri. It's awesome in every bit. Not only does it have a state-of-the-art voice recognition, it's also packed with super smart artificial intelligence that supposedly allows you to ask crazy things things such as:

"Can you remind me to call my wife when I leave the office?"

Another very legitimate question.

And there's a strong resemblance there. Both requests are really abstract and probably require quite a bit of computational power to be understood by a program. They have nothing to do with mathematical or social ranking currently used by Google (search), they are all about Artificial Intelligence and semantic interpretation. And while Google currently doesn't provide (or at least market) services that would be able to understand such sentences, Apple does.

I've noticed quite a few articles saying concepts such as Siri are the future of search. It's obvious artificial intelligence will play a big role in this segment. Apple's already in. Even if their technology is not superior to Google's, who is also working on embedding AI into search, it's fully available today, and everybody knows it. Google should really be careful with such statements concerning their core business, Web search. Specially if they are competing against the marketing wizards of Apple, who know how to sell things even if they don't fully work.

Promoting a technology you don't have and your competition does? Stupid consumers such as myself might do something stupid.

UPDATE (5.12.2011): You can join the discussion on HackerNews.

Television and Social media? How did my recommendation engine miss this connection?

Sun, 27 Nov 2011 14:58:20 GMT

November has been a great month for this blog. For the first time in history, I managed to get more than 1.000 unique users on two different blog posts in a single month. Which is awesome, thanks! The first post was about the TV show Dexter and its Facebook game Slice of life. The other was about Slovenian TV show Soočenje and its buzz on Twitter. Just two posts, nothing special, right? Wrong. It's really obvious, but I missed it somehow. Both posts are talking about combining television and social media, silly me! I can't believe I failed to see it, but I did, and so did my blog. Not that it really matters anymore. You know those fantastic coincidences that happen sometimes and put everything into place? This story is full of them.

Function

Some of you may know this blog has an internal recommendation engine that calculates the correlation between different posts based on shared tags and their frequency, offering related reading in the bottom. It missed the connection. Others may know I'm a bit obsessed with cross-referencing my posts, which I do manually. I missed it too. Perhaps Facebook and Twitter aren't as similar as I would like to believe, but I'm putting my bet mostly on the different concepts of combining television and social media. The Dexter case was about entertainment, gaming and story-telling. Pop TV's case was about politics, news and ordinary people co-creating content. Different problem, different tags, but still, the strong relation between the two is very much there.

Try

The first funny coincidence was a blog post by @anejmehadzic written a few days after mine, discussing the general possibilities of a symbiosis between television and social media (in Slovene). The post provided enough insight to make me see what I missed. TV shows using Facebook, Twitter and YouTube to provide additional content to viewers was really something in between the two cases of mine. Revelation. At this point I knew I missed the connection myself, but how did my very smart algorithm also miss it?

Catch

The next lucky coincidence was a lecture on wwwh happening yet a few days later. @zbrchka was talking about transmedia, a term I haven't heard about before. I thought multi-platform or cross-platform could be concept that connected these two blog posts, but transmedia feels so much better. Transmedia is a technique for creating integrated content for different mediums, just the thing what I was looking for. Something that's becoming so important it deserves exposure on this blog too.

Finally

With the new gathered knowledge, I made a new tag Transmedia, putting it on both posts, besides the one about Šport TV tweeting about the basketball championship. It worked like a charm. Since this tag is used so rarely, it dominated the recommendation engine, and to my great relief, all three posts gained the correlation they require to be listed as related content one to another.

The blogosphere kicks ass, since bloggers are mutually inspiring each other and moving things forward. Wwwh is a great place to hang out and share knowledge and experience. My recommendation algorithm is awesome, fully working as expected. And those lucky coincidences are a thing that make this existence an interesting place to be visiting. Everything is just the way it should be.

* try-catch-finally is a an exception handling syntax used in some programming languages.

The great aquarium cleaning dilemma: should you be removing or replacing water?

Tue, 01 Nov 2011 18:44:18 GMT

Everybody that owns an aquarium probably came across this decision at one point. The water is filthy and needs to be replaced. All you have is a jar. And you ask yourself: should you be emptying the aquarium first, adding new water later on, or should you be replacing filthy water with clean water? The first choice seems more rational, but sometimes you can't fully empty the aquarium (e.g. you have fish), and you need to do more runs since you're not taking water both ways. The other option seems interesting since you're efficient both ways, but at the same time you're taking back fresh water mixed in the aquarium. So, what should you do?

The situation

In reality, you do have other options. A water pump, a larger intermediate basin or other things that can make this task easier. But believe me, sometimes you don't have the time to do it properly and you just want to clean the water a bit. And that's when you'll wonder what to do. It happened to me, and that's why I've made myself a model that would answer this question, a model that would determine the breaking point (where both options are equally effective) between the two techniques.

The model described contains the following parameters:

the aquarium volume (V - volume)
the jar volume (d - change)
number of two-way runs (x)

And it looks something like this:

The aquarium cleaning situation

The initial model

The original view I made in Excel is based on simple mathematics, where each one-way run is represented by one line in the table. Adding and removing water both-ways constantly reduces filthiness, but makes each additional run less effective. On the other hand, removing water first and adding fresh water later takes more runs (since your jar is empty in one direction, you require four one-way runs to replace an additional unit), but you're not removing clean water. Here's a preview of how this looks:

The basic calculation

The advanced model

The basic model is more understandable, but not appropriate to make a full mathematical equation. That's why I made a second model, which is based on two-way runs and exponential functions. Here's what I got:

The advanced calculation

The results

The results show that if you don't have much time, it's better if you're replacing water, taking it both ways. This accounts for faster cleaning, which is slowing down in the long run. But the point is that no matter what the parameters are (aquarium and jar size), the breaking point happens more to the end of the cleaning. So, if you're not prepared to fully empty the aquarium, it's generally better if you use the the first option, removing and adding water at the same time.

The graphical results displaying water filthiness based on number of two-way runs for different jar and aquarium sizes

The breaking point equation

The results are there, and I managed to make the following equation which would calculate where both ways are the equally effective:

This makes calculating the number of runs so complex it can't be calculated in basic Excel (because of the Lambert W or product log function).

But luckily, you can use Wolfram Alpha to calculate the breaking point of two-way runs by entering the aquarium and jar size in the boxes below. If you're prepared to make less two-way runs than the result, take water both-ways, otherwise, one-way.

Aquarium: Jar:

If you want to play around a bit, you can even download the complete model in Excel format.

Conclusion

Perhaps there's an easier way to calculate the great aquarium cleaning dilemma, which speaks in favor of water replacement (taking water both ways). There's also a chance I've made an error somewhere (please let me know!). But the results seem correct, and I hope you find all of this amusing and / or helpful when you need to update your water. Aquarium cleaning will never be the same again.

Update (26.11.2011): While this may have been a fun experiment, I've just read it's not that smart to fully remove water in the aquarium, since it disrupts the balance of bacteria.

Facebook vs. Twitter, Part 3: The phase of unification

Thu, 13 Oct 2011 08:22:32 GMT

I started writing about Facebook and Twitter because I saw these two services as the most impressive players of the social age. I received a lot of comments about the two of them not being comparable, which I disagree. They are the biggest global Web 2.0 platforms (LinkedIn successfully went public and has a lot of users, but it's hardly a platform) and two of the ten most visited websites in the world. They are social trend-setters, both super advanced on technical and conceptual levels. They are a lot, but with the latest sets of patches, they are also becoming a lot alike.

Originally, Facebook wanted to be a social network. On the other hand, Twitter wanted to be a news network. But seeing what's happening these days, we can ask ourselves: is it rather the other way around? Indeed, Facebook and Twitter are finally entering the phase of unification. Let's begin.

Two way integration

Twitter now officially supports posting to Facebook, while Facebook now officially supports posting to Twitter (for now, Pages only). Until recently, you had to use other services or install apps to fully connect both accounts, but these days, direct integration it's pretty much trivial. The wall between the two giants is obviously coming down. And they both noticed the need to recognize each other to proceed.

Facebook adds following

The symmetric relationships (friends) on Facebook made it what it is - a social network for keeping in touch with the people you know. The same goes for Twitter, the asymmetric relationships (followers) made it what it is - a news network where you can follow people you are interested in. This fact was one of the biggest differentiator between the two social services.

Facebook soon noticed they will have to somehow go beyond that, for the sake of enabling more accessible information to the masses and enabling unprotected, crawlable real-time data without privacy. They've started with Pages, intended for brands, which users can like and follow. But recently, like I predicted more than a year ago, Facebook introduced subscriptions for personal profiles, where you can follow people without the need for them to confirm you back, and they can post public updates. What an interesting turn of events.

Facebook adds a real-time stream

Facebook put a lot of effort into developing the EdgeRank, which recognizes the level of connection between two people. The previous version of Facebook had Hot and Fresh streams, the first one being based on the amount of activity and EdgeRank vicinity, and the other one on recent activity. Today, the main stream is a mixture of both, while a new feature was introduced. The so-called Ticker in the right sidebar, showing real-time activity from all your friends. Your own little Twitter inside Facebook, which they plan to evolve even further, adding automatic updates.

Twitter starts to close, adds multimedia

Traditionally, the Twitter app ecosystem was built around (outside) Twitter, while Facebook's app ecosystem was built inside Facebook. But Twitter's policy is changing. They've already acquired one of the biggest Twitter clients TweetDeck, and said they will prevent new Twitter clients from being developed. They've also added a multimedia library to each profile (feeding also from external services) and enabled an internal service for sharing photos. Bad news and a stab in the back for Twitpics, Yfrogs and such. Smells like Facebook.

Twitter adds activity streams

Social networks are all about profiles, streams and interaction. Twitter used to be plain, providing only simple profiles, basic tweets stream, replies and retweets. But the newest addition also includes real-time activity streams, where all retweets, replies and favorites are gathered in one place. I haven't gotten it yet, but people are saying Favorites are the new (Facebook) likes. Twitter is becoming very much a social network, trying to boost up it's social graph.

Conclusion

There are even more cases like this, but these are the most significant. I hope this trend won't continue much further. It was the differences between Facebook and Twitter made each one interesting and useful in it's own way, but I guess things like this are inevitable. Like operating systems or browsers, competitors are constantly copying each other's features and solutions, which actually proves mutual recognition. Why would social services be any different? It seems Twitter and Facebook are finally mature enough to enter the phase of unification.

Check out the complete Facebook vs. Twitter series.

Solid-state drives (SSD) are the biggest win in hardware ever

Wed, 21 Sep 2011 08:01:42 GMT

Guess what? My new computer has a Solid-state drive. In case you don't know what I'm talking about, that's the new super fancy high-tech computer storage unit. And I must say it's so amazing it's almost silly. Extra fast boot times, complex applications opening up in seconds, less power consumption and no humming from the hard disk. What else could you wish for in your new sexy computer? I love it, and I love it so much I had to tell the world.

In my laptop career, I went from from Centrino to Core 2 Duo to i5. Never noticed much difference in performance. Well, with the last one, I also got the mentioned SSD and it blew my mind. And I was wondering, besides the famous Samsung SSD in RAID movie, I haven't heard much about it, nor seen impressed geeks writing poems about its awesomeness. Sure, people are testing how faster it is and arguing about the fact if it consumes less battery or more. But who cares, what you really want to know is if it makes a significant difference or not.

So trust me on this one: nothing will make more difference in personal computer performance than a Solid-state drive. By miles. Lightyears even. And while I know they are expensive as gold, I still think you should consider getting one with your new computer. They are worth every penny. If you're not a hardcore gamer, make movies or process calculations with thousand of parameters, you really don't need an extremely capable processor and a state-of-the-art graphic card. Memory, you need that, but it's cheap. What you need the most is a Solid-state drive. Casual user or one of the mentioned.

There's always a downside. After some time, Solid-state drives supposedly start to behave a bit slower and are still not as reliable as hard disks. Luckily, most of these problems have been taken care of with the current generations, so we'll see what happens. At this point, all I can say is that I'm very very impressed and think that SSDs deserve much more than just this post. They're pure win. And another item for your wish list.

The art of internal hyperlinking

Sun, 04 Sep 2011 21:00:02 GMT

Everybody wants traffic on their blog or website. Direct traffic. Referring traffic. Search traffic. All good in their own way. Direct traffic means having a strong brand. Referring traffic means having a strong network. Search traffic means having a strong team. The first two are hard to influence, but search - that's the one you can influence the most. And even if search technology has changed a lot in the past few years, with mathematical algorithms slowly getting replaced by social ones, old school search engine optimization can still make a difference.

Let's say you already have good content. And you have it technically optimized and search engine friendly. Like, Tweet and +1 buttons implemented where applicable, since they are more and more important for search result rankings. All good, but you might have missed something. Since it's very hard to persuade other people to link to your site giving you a higher PageRank, you just have to do it yourself. I'm talking about internal hyperlinking, where you cross-reference the content you already have. Google loves it so much that their official blog, besides influential blogs such as Mashable and ReadWriteWeb, have main headings referencing itself.

I approached this issue from two different angles. Sometime in October 2010 I took the time to systematically cross reference all my blog posts. It took me a lot of time, but I think it was worth it. Besides, I developed a "similar content" module, which additionally does it for me. The results on the diagram below, which display search-based traffic on my blog, are not real proof of that fact, since the experiment was not fully scientific; there are many other SEO factors, additional content and optimizations that were put into the equation. But still, most energy was put into content cross referencing and the trend looks pretty good.

Traffic to http://stritar.net via search engines

Since internal hyperlinking is an ongoing process, you have to have a good overview about the content you own, something that provides you with the complete picture. I used a plain Excel file, grouping my posts into general categories and started drawing arrows representing hyperlinks. At this point, it's already become a bit of a mess, so I'm thinking about moving to a stronger diagram-oriented software, but I think you get the picture.

http://stritar.net internal hyperlink structure

Internal hyperlinking can help, and it's something you can do even if you don't have a lot of technical skills. Just a lot of time, but we're in a recession anyways. Now go for it and let me know about the results.

Please help me upgrade my Twitter bot

Tue, 30 Aug 2011 20:33:12 GMT

Half a year ago I decided to make something out of my Delicious bookmarks. The magazine-style display inspired by Flipboard wasn't enough, I wanted to publish these links somewhere outside my chronolog, somewhere on Twitter. So I made a bot. It's doing quite well, posting like mad, but it's really not where I want it to be. Until now, it made about 3.000 tweets (around 500 per month), but has only 67 followers. I know my taste in content is a bit obscure, but still, only 67 followers?

This calls for an upgrade. And since I won't change my interests and bookmarking habits, something else needs to be done. That's where I need your advice. Crowdsourcing the concept and stuff.

The brand

This is probably one of the greatest fails of the project. At first, I thought of it as an extension of my blog. Hence the account @stritar_net, the name Stritar's chronolog, together with the description it has. Should I rename it and try to make it a standalone "brand"? Should I openly say it's a bot (Stritar's bot or something)?

The selection

The bot currently posts ALL my bookmarks to Twitter (as mentioned, around 15 per day or 500 per month) without any selection. But it could be done. Since my bookmarks are originally tagged, it could leave out those with too few tags (since I use the same method of counting tags to determine the initial weight of content for the magazine). Or specialize in specific segments according to tags. There could even be more of them bots. What do you think?

The frequency

Currently, the bookmarks I make go into a queue. I hate those Twitter accounts that post 10 tweets in 5 minutes and go silent for a day. I wanted to make it more smooth. So the queue always knows how many items it holds and adapts the frequency of posting according to it (less bookmarks in queue mean less frequent tweets). But that produces the situation where most of them are already a few hours or days out of date when they are published. A higher publishing frequency would solve some of it, but it opens a great dilemma: what's the lesser evil, over-spamming or being out of date?

The order

The order of bookmarks posted on Twitter is determined by two factors. The number of tags and the date they were published. More tags equals more importance. Older bookmarks get published sooner, otherwise they would get even more out of date. Should I do it the other way around and post more recent links sooner? This would make some of them more interesting and up-to-date, but other worse. Breaking content or consolidated content?

So many decisions… In this case, the best way probably doesn't exists, but trade-offs can always be decided for the better. Please let me know what you think, your help would be more than appreciated. I could help back if I can.

Determining if an element is in the last row of a table

Mon, 25 Jul 2011 11:57:52 GMT

Once upon a time I stumbled upon a problem, where I needed to calculate if an element is in the last row of a table. Here's the scenario: you have a number of items, which are put in a table from left to right. When the row is full, the items continue in the next row. Imagine an airplane or a theater where people start sitting front-left and continue to the right until they run out of space, then going to the next row and so on. Now we want to know which people are sitting in the last of the populated rows.

A weird problem, but hopefully I will be able to show you some cool results produced by this algorithm someday (yes, it's usable).

The equation has 2 parameters: the total number of elements and the number of columns in a single row. There are a few ways to do it, using division with remainders (the modulo operation). The simple way would be comparing the total number of rows with the element's current row. Another one would be to calculate the number of elements in the last row and see if our element is in those last few. While both seem logically easy, they actually suck, because they contain exceptions (some states need to be handled specifically - if the last row is full or not).

That's why I went for the ultimate way, using abstract mathematics which requires real magic, and since it would be too boring to explain it, just take it if you need it and don't try to understand - to be honest, even I'm not perfectly sure how I did it.

The number of rows way:

lastRow = ((Math.DivRem(currentElementIndex + 1, numberOfColumns, out remainder1) + Convert.ToInt16(remainder1 > 0)) >= Math.DivRem(totalElements, numberOfColumns, out remainder2) + Convert.ToInt16(remainder2 > 0))
* Math.DivRem returns the division result and the remainder, since both are required for the calculation

The number of items in the last row way:

lastRow = (totalElements - (totalElements % numberOfColumns) - (numberOfColumns * Convert.ToInt16((totalElements % numberOfColumns) == 0)) < currentElementIndex + 1)
* % is the modulo - remainder from dividing

The ultimate way:

lastRow = totalElements < ((numberOfColumns + currentElementIndex) - (currentElementIndex % numberOfColumns) + 1)
* % is the modulo - remainder from dividing

Mathematics is awesome.

UPDATE (27.7.2011): Silly me. Overwhelmed by "The ultimate way", I missed the opportunity to simplify "The number of rows way". I guess this is one of the cases which explain why counting starts with 0 instead of 1 in programming.

The improved number of rows way:

lastRow = Math.DivRem(currentElementIndex, numberOfColumns, out result1) >= Math.DivRem(totalElements - 1, numberOfColumns, out result2))
* Math.DivRem returns the division result and the remainder, since we need the number rounded down.

Is Delicious aiming to become the next Twitter?

Thu, 26 May 2011 17:03:41 GMT

The bookmarking service Delicious has had an interesting life. It was one of the first social services available, later bought by Yahoo and almost canceled, then being sold to Avos about a month ago. Avos was founded by the same people who've created YouTube (Chad Hurley and Steve Chen), and these guys obviously know what they're doing. A few days after acquiring Delicious, Avos also bought a social media analytics startup Tap11, and here's what they had to say about it:

"Our vision is to create the world's best platform for users to save, share, and discover new content. With the acquisition of Tap11, we will be able to provide consumer and enterprise users with powerful tools to publish and analyze their links’ impact in real-time."

While some bloggers think Avos will start competing against Google and Salesforce.com by analyzing social data, I can imagine a different strategy may be plotting. Let me explain.

Delicious was always ahead of its time, but did not really make it to broad mainstream. It allows online bookmarks, which you can tag, bundle and keep in a library for later use. It knows asymmetric relationships, so you can check out bookmarks by the people you follow. The bookmarking engine is really powerful, but something was missing. Delicious' biggest problem is its social layer - too weak and of secondary importance. In the mean time, other services such as Digg, Reddit and StumbleUpon took their place on the web and added communities and different types of recommendation to link sharing. And of course, there's Twitter, the current ruler of content and real-time.

Actually, Twitter is slowly becoming a content sharing platform rather than a microblogging platform (I guess microblogging should involve content creation, not sharing). But while your links may bring you audience, they are not categorized and useful to you. Still, most people use Twitter that way, and even authority-measuring services such as PeerIndex and Klout encourage you to share links, because that's what Twitter is all about and that's what will make you influential.

I'm not saying Twitter is not useful, it is very much useful. But imagine having a solid bookmarking platform, very useful for the person who uses it (save). Add a generic social layer of friends and followers, a few comments, perhaps something similar to what YouTube has (share). Now add a hard core mathematical layer which is able to calculate what you'll like based on what you already liked (discover). What you get is something that could be very special, something that could compete even with Twitter. And it could be happening right now in Avos' laboratories.

One guy said that the age social sharing is coming to an end. I think not, there's loads of information thirsty people surfing the web. What's really missing is a new innovative and powerful platform, something useful in many different ways, for keeping, dispatching and receiving new, personalized content. Delicious 2.0?

Trademarks and logos are the property of their respective owners.

A few thoughts on content categorization. No surprises there, less is more.

Fri, 29 Apr 2011 12:26:00 GMT

Since I've started collecting bookmarks using Delicious, I've put a lot of effort into their categorization, organizing them in such a way their browsing would be as simple as possible. The service supports two level categorization (tag – bundle) which helps to control massive amounts of links people have gathered. But it's the experimentation with different structures that gives real insight into content categorization, and because this topic was already mentioned and discussed a few times on this blog, it deserves a special mention. Let's begin.

Categories vs. Tags

Observing other blogs, I've noticed a lot of them use both Categories and Tags. While I can understand the SEO (Search Engine Optimization) benefit in having as many different entry points (landing pages) as possible, I don't see any other added value in using both. From the logical point of view, they do the same (categorize content), but on a different level. Here's where tag bundles come handy. With my bookmarks, I use tag bundles such as Wibe, Science, Brands, Work, etc., to combine different tags into groups according to their qualities. And aren't Categories and Tags just another form of the same thing, just two different tag bundles? Perhaps not, but that doesn't change the fact one is probably redundant.

I still see cases when Categories are used as single items (one post is filed under one category), while Tags are always used as multiple items (one post can have many tags). This corresponds with the technical 1:N and M:N database relationship, and even though the second is a bit more complex to create and maintain, it provides much more flexibility. Hierarchy vs. matrix.

Less is more, and intersections rock

Another thing I've noticed is that people use a lot of different tags. Too many to handle. I try to keep the number of tags as low as possible, working rather with intersections of tags (e.g. marketing + twitter) than looking for specific tags, used only a few times. I made a quick calculation on how this works, estimating a model with 10.000 contents and 200 tags, which corresponds with my situation on Delicious:

10.000 contents, 200 unique tags, average 5 tags per content
10.000 contents * 5 tags = 50.000 total tags
50.000 total tags / 200 unique tags = 250 occurrences of each tag (contents per tag)
5/200 probability of first tag * 4/199 probability of second tag = 1/1.990 (0,0005) probability of two specific tags on a single content
or (200! / (2!*(200-2)!) = 19.900 unique combinations of two tags; one bookmark with 5 tags allows 10 pairs of bookmarks, making a combination's probability 1/1.990
1/1.990 * 3/198 = 1/131.340 (0,0000076) probability of three specific tags on a content
Result: on average, 5 contents out of 10.000 will contain two desired tags and 0,07 three tags

The model is built on the assumption that all tags are spread evenly, which is far from reality, but you get the picture, the number of contents with multiple tags is pretty low. But if you lower the number of unique tags (e.g. 150 tags instead of 200 would raise the number of contents with a pair of tags from 5 to 8,9) or use the same tags more often (e.g. 6 instead of 5 tags per content would raise the number from 5 to 7,5), the results get even better. Basic mathematics is a powerful tool, and intersections with two, three or more tags are definitely the way to go.

Applications

I've made a few applications using the techniques mentioned. For general Categories of this blog, I used a combination both, having Categories behave like Tags, using a few of them as possible (but attaching many on a single post), displaying them as a tag cloud (bottom of the page). I used a similar approach on my iTunes library, abusing song Comments to act as Tags for advanced smart playlists. And some time ago, I developed a simple engine for related content, based on occurences of different Categories / Tags on my blog posts, acting both as an additional feature for readers, as a tool for internal hyperlinking, used for SEO.

These are a few cases which display the power of simplicity, using as little data as possible to create a lot of information. And while I know this is hard to do, I must continue to pursue this philosophy, may it be in software development or blogging (I ironically failed with this one). Things that are similar on an abstract, logical level, should be the same on the technical level. Try it, you'll be amazed by the results which will present themselves.

Can you believe Watson got the question about Slovenia wrong on Jeopardy?

Wed, 09 Mar 2011 07:33:12 GMT

Slovenia made it to the spotlight again, for the first time after the soccer world cup (when Slovenia was trending topic on Twitter and top search on Google). This time, it happened because IBM's supercomputer Watson competed against human champions in the famous TV show Jeopardy. IBM's computers are known to destroy people in various challenges, Deep Blue beat the world champion Garry Kasparov in a chess tournament in 1997. But chess is simple for computers to play, because it is pure logic and mathematics – the capability of a player is determined by the number of operations and actions it can calculate in advance. But a quiz is a totally different story, where the biggest challenge is semantics – understanding the meaning of words.

In 1950, Alan Turing, one of the greatest pioneers of computing introduced the Turing test, a methodology that could separate humans from computers using a set of questions, some of them formed in such a way computer wouldn't be able to understand and answer them. There are many questions which can't be answered with pure logic, the one I remember from high school goes something like this:

"Jack attended Sally's party, bring a doll. What was the present?"

The catch is in the connection between party – (birthday) – present – doll, which can't be noticed without abstract thinking humans are capable of. And today's computers still face the same problem - even though Watson dominated Jeopardy, it failed miserably on the following question about Slovenia:

"As of 2010, Croatia & Macedonia are candidates but this is the only former Yugoslav republic in the EU"

Watson's computing capabilities and knowledge banks are huge, but a question and an answer so obvious to humans presented a huge problem. Watson surely knows which countries are EU members, but it obviously didn't understand the question, thinking it was asked about which country would be next to start negotiating for EU membership, answering Serbia. The right answer was, of course, Slovenia.

The video is also fascinating from the cultural point of view – and extremely creepy. Those who have watched (or read) "2001: A Space Odyssey" may have experienced a slight shiver and carefully waited if Watson would say it: "Hello Dave". Others might have enjoyed this science fiction presentation, but besides Watson's obvious advantage in being the fastest to answer the question, it's clear that computers are still far away from being intelligent. And hopefully they will stay that way.

I've developed a magazine based on my Delicious bookmarks. And a Twitter bot.

Fri, 25 Feb 2011 07:34:38 GMT

Last week was marked a great social achievement of mine – I managed to bookmark my 10.000th bookmark on Delicious. A lot of people have 10.000 tweets, but not many own 10.000 bookmarks, fully tagged and classified. I've been collecting these since December 2006 (probably one od my first 2.0 addictions), and they are becoming one of my greatest possessions – knowledge is the ultimate collection. Hopefully Delicious won't get shut down or left behind, so I will be able to continue with this obsession.

I've made a few interesting mashups with them already; as my most frequent actions online, my bookmarks represent the core of the chronolog. But things are changing fast and the desire for presenting information is moving into a new dimension. In case you didn't notice, tablets are mainstream, and the media industry already hopes they are the solution they've been waiting for. The Daily, which was released a few weeks ago, is the first no-print, tablet only magazine available (for iPad, Android version is coming), and other applications, based on social curation, such as Flipboard (which is amazing and inspired all of this) and OnSwipe are revolutionizing the way we (create and) consume content.

I find this evolution of displaying information fascinating and a bit ironic - going from newspaper form to blog form and back - but I've decided to play along, developing a magazine based on my bookmarks. Using the Html Agility Pack library for asp.net I managed to extract an image and a few paragraphs from each URL I bookmarked, using them to form a magazine. The number of tags I put on each link determines the initial weight, which is additionally modified with your views and likes, allowing it to constantly adapt its shape. Those links which are stronger, are displayed higher, have a bigger picture and more text which makes them more visible.

While contemplating about the potential of different APIs and all the pages I bookmarked these years, it occurred to me I could also make a Twitter bot. All the bookmarks I save are now getting automatically shortened with bit.ly and posted to Twitter, with a bit of artificial intelligence. The speed of posting is determined by the number of items waiting in the queue, adjusting to the frequency of my actions. It still has a few problems, but they are only appropriate for a prototype, whose posting algorithm still needs to be technically and mathematically improved.

The life cycle of my links I like has become quite a ride, as you can see in the diagram below, similar as the evolution of the Web, transformed by social, mobile and the upcoming domination of the tablets. And while the major players are able to spend millions on the development of new and creative solutions, small players such as myself can only play along. Luckily, I like to play.

The path each one of my bookmarks makes in its lifetime.

Why an actual Facebook phone could kick ass (with mockups)

Sun, 09 Jan 2011 19:28:34 GMT

The question if Facebook will start producing it's own mobile OS on top of Android made a huge buzz a few months ago. Technology authorities such as TechCrunch and Mashable gave us diametrical coverage about it, the first claiming the rumor is true, and the second denying it. Needless to say the mobile market is one of the fastest evolving. Since 2007, when Apple supposedly revolutionized the mobile telephone by introducing the first popular tablet smartphone without a keyboard with an app market, things didn't change much, but in 2011, Facebook has a great chance to reinvent the phone again.

Today, the world is "going social". The transition to 2.0 has touched most aspects of our lives and heavily influenced software development, some companies are introducing social browsers, and others such as Neolab are trying to put social into enterprise IT. But the most social device of them all, the mobile telephone, despite high competition and increasing hardware and software capabilities, was left behind. Sure, the social potential in mobile contacts was noticed, but we haven't seen it happen yet. So, if Facebook actually gives this thing a try, could we finally see a real social phone?

One of the biggest problems with today's mobile operating systems and their user experience is that they are still built around services rather than around people (contacts). You have your app for calling, your app for messages, your app for mails, apps for different social networks. Different channels with enclosed streams rather than one giant stream that would display all the communication and interactions with a specific person. Android and Windows 7 Phone did enable Facebook contacts syncing with direct links to profiles and some integration, and there are third-party apps that are trying to achieve this (SocialPhone, Phonebook 2.0). Nevertheless, I still made some mockups of my own about how I envision the social phone OS of the future (since I'm more aquainted with the iOS, I worked on that), something that is destined to happen one way or another.

User stream

The most important feature I miss about the current generation of smartphone operating systems is the complete stream of activity and interactions from a single person. All classic mobile services (call, message, mail, calendar, ...), combined with social services (Facebook, Twitter, Foursquare, ...) in a single feed. Here's a mockup of how this could look, the icon represents the service, the arrow represents the direction (in case of public posts, which are not between two people, there is no arrow, since it's an action without target instead of a reaction). All services are intended for communication, so why are they kept separated and treated differently?

Single user stream, displaying information from different sources and services. Similar features already exist in some apps.

The contacts

Now that we have an integrated repository of all our social interactions with a single user, we could reinvent the address book. Each action could have it's weight (e.g. a Call would be much stronger than a Twitter reply), and the occurrence of social interactions with a user in recent time period could determine the probability of needing that specific contact (an upgrade to "recently contacted", available today). To make things even more useful, users could set the preferred time period using a slider. Those who have seen how sexy iPhone icons behave while being rearranged, can probably imagine the fancy shuffle of profile pictures upon this activity.

Contact grid, where people are recommended based on the number of social interactions in a specific time period.

The activity log

To top it off, I've made a mockup of the complete activity log, which could combine all owner's social activities together with the interactions on a mobile device and other services. Again, the icons represent the public actions (shown with a service logo) and the interactions (shown with a profile picture), together with the direction of the reaction. This blog does something similar, combining different social activities into an unified stream, but it's still mostly one way - my posts on different social services. But combining one way posts with two way actions/reactions/interactions could provide the component that could actually make the phone capable of portraying the most perfect social stream of its owner. Specially since regular phone activities, such as calls and messages are as social as you can go, but they exists only on the device and the carrier.

Log of all user's activities on the phone and on different Web 2.0 networks.

These are just a few ideas, I've been thinking about trying to pack them in an app, but I really don't have the time to go for it, and similar ones are already available. I also think that this concept of a social phone should be built into the core of the OS, because the phone would need to be completely integrated with and authenticated into different Web 2.0 services (not only single apps). This would make these features available inside other apps, and setup and synchronization would require less hassle.

Facebook, the king of social, is currently the most perfect candidate (and perhaps the only one capable) to make something like this work, so I would really love to see it come alive. It's questionable if they would allow competition like Twitter or Foursquare inside it, but other software giants would surely need to follow the concept and in the end, make it right. I want a social phone!

Are we on the verge of a full-scale cyber war?

Wed, 08 Dec 2010 08:15:15 GMT

The whole world is talking about WikiLeaks, and the drama has reached its peak with the arrest of Julian Assange. The leaked diplomatic documents are obviously a major thing, something that could change the world as we know it. The main battleground of this conflict is cyberspace, where troops of different armies are already fully ready for combat. WikiLeaks.org is currently offline, being a constant target of attacks of all sorts. But the civil initiative is striking back, putting hundreds of mirrors online (even Slovenia has one!), and hacking the bank that closed Julian's account. Even the good boy Twitter is behaving weirdly, denying accusations of censoring #wikileaks as trending topic.

We've seen similar cyber battles before. The Anonymous (who are also defending WikiLeaks) had their fun with the Church of Scientology and Google had its showdown with China. But WikiLeaks is a probably a few levels higher. I always said that internet could be the main tool of the next revolutions, because information travels really fast and it's almost impossible to prevent it from spreading once it's out. But the problem with the Wikileaks situation is that it may be too much for our society to handle. Diplomacy was exposed, which means the current political situation, already in a weak equilibrium because of the upcoming shift in global power, will struggle to digest the information presented by WikiLeaks. And there's more to come.

All of this is probably history in the making, because we have never seen an event of this sort going so mainstream. And while we wondered if the blind woman's prophecy about World War 3 starting in November 2010 was about the Koreas, we might have missed the point. This could be the start of a civil revolution or even a real war, in which governments would surely have problems identifying hackers and double agents inside their ranks. I have mixed feeling about it myself, it's simply too overwhelming. But the general reaction of people, now matter how dangerous and foolish the revelation may be, clearly shows disappointment and lack of belief in the current "democratic" system. Are you ready for a new world?

The chronolog now understands connections between content

Wed, 03 Nov 2010 21:20:56 GMT

I once made a promise that I will try to incorporate as many interesting features as possible into my blog. My previous development sessions were based mostly on interactions of readers with the posts, the peak of it being the Hot on the chronolog algorithm. But now, as the chronolog finally reached critical mass in the amount of content it operates with, the time has come to do something new. The next step is focused on a different functionality, and a few days ago, the chronolog received an algorithm for recognizing relationships between different blog posts.

The connections

The whole concept is based on the occurrences of categories (which are actually tags) on different blog posts, the most obvious being the number of the same tags two different posts share. We did something similar on a web portal we launched a few months ago, and it works pretty well. Sure, the proper way to do it would be using real text mining, where the strength of the relationships would be based on meaning and occurrences of words and external hyperlinks in a specific post. But in this stage, I'm keeping it simple: if two posts share a lot of tags, they appear more related.

The weight

Since some categories (tags) are used more often, they appear in many posts, making these posts too heavily related with each other. The number of categories attached to a single post also varies, giving a post with many tags a much stronger chance to appear as related to another. Therefore the general equation contains two modifiers, which are giving weight to each shared tag between two posts.

Categories that appear only a few times globally, have more weight, because they represent a more scarce and therefore a more interesting and stronger connection. This takes care of the tags which are used very often, making them not too dominant. On the other hand, the weight of each tag on a post drops with the total number of tags the post has, so those posts, which have a lot of tags, don't become every other's related post. It may sound confusing, but it's probably a bit simpler to develop than to explain.

The results

I was actually quite surprised about the result the algorithm makes (which you can now see on the bottom of every post). As I was playing around a bit, observing how the calculation behaves and playing with constants, I actually found some interesting connections between posts which I didn't notice before. The engine finds quite a strong relationship between the post about using Web 2.0 logos in TV commercials and the one about Round browser icons, both of them being design clichés. The case of Microsoft and Google going social also made it strong, as the two posts are describing the struggle of two technology giants trying to adapt to the new situation. I could go on and on, but than you would probably just say I was doing SEO too hard.

Search Engine Optimization (SEO) is actually another hidden benefit of the feature, something that occurred to me after I've already finished working on it. Google likes it if you have your content internally cross linked, so what better way to do it than to have automation take care of it. So until SEO dies, this new functionality is actually a double win, because the chronolog became more optimized for crawlers and hopefully more useful for the readers. Even though most of you probably won't even notice.

Apple's strategy of becoming a content provider might simply be ingenious

Sun, 19 Sep 2010 15:38:22 GMT

Apple is an interesting corporation. Some love it, some hate it, but the fact is, Apple has become the biggest technology company in the world. An interesting turn of events, from a company that nearly went bankrupt a few decades ago, to a player that we know today. Looks like Steve Jobs really is one of the greatest visionaries of our time, as his comeback in 1996 together with the introduction of the iMac and the iPod managed to turn things around for Apple. Looking at these facts in 2010, the iPod may turn out to be even more important than it seems, creating a digital music revolution and providing the foundation for Apple's strategy of becoming the world's dominant commercial content provider.

Apple was always known as the great innovator, and all of its products and services(!) strive towards perfection. This vision of making things people want and need (or don't need but still want anyways) has moved beyond hardware and software, towards a new direction – providing both free and payable content in any form. And because of the loyalty and fanaticism of their customers, they are one of the few that are actually able to make this work on a global scale. And be aware content is more than just actual news, it's multimedia (music, videos), it's software, it's books, it's just about everything, and Apple was the first to notice this potential and put it into action.

It all started with the iPod (2001) and its biggest sister, the iTunes. Today, iTunes is the largest music retailer in the US, with over 10 billion songs purchased. With the introduction of the iPhone (2007), its biggest sister, the App Store became one of the largest software marketplaces in the world, selling more that 6 billion applications in two years. The next gadget, the iPad (2010), was launched with a bigger sister too, the iBooks, which is currently messing up Amazon's Kindle operation, even though the success of this story is still hard to estimate. A few weeks ago, Apple also introduced the new cloud powered Apple TV (2010), which will enable selling (actually renting) fresh HD movies and TV shows using the iTunes platform. And to complete the circle, there are rumors Apple will also provide a subscription service for newspapers, supposedly the iNewsstand, which could actually become the biggest of them all.

So, how did a "personal computer" company come to this position? If you ask me, the answer is simple – Steve Jobs. This visionary was able to predict where the future will go and if you look at the situation today, when this business model is proven to work, it's really trivial. When personal computers were on the rise, it was obvious business and sales will be in hardware and software. But in the present information era, where billions have access to the web, overloaded with information, it's almost obvious that business and sales will be in providing content: data and information in all forms.

Besides Apple TV, Ping, a social network on top of iTunes, was also introduced on Apple's last "mass". I won't speculate on how cool it is, because I haven't tried it and am not sure if I will. But the fact is that iTunes has 160 million users and generates a lot of revenue. Ping actually represents a first integrated wide scale social market place, which probably is the future of social electronic commerce, so it will be interesting to see what will happen. No matter how cool, adding a Facebook Like button on Amazon simply isn't real social commerce.

That's Apple, once a computer manufacturer, today a diversified technology and media corporation with great know-how in electronic commerce and providing content, while pioneering (again) in social commerce. Besides owning a complete set of platforms for providing and selling different types of content, many of them market leaders, they also offer beautiful hardware and software infrastructure for their implementation. Something we will be seeing in most living rooms of the future?

Facebook vs. Twitter - Part 2: Privacy and real-time web

Thu, 13 May 2010 17:11:14 GMT

After my previous post comparing Facebook and Twitter I received a few comments about how Facebook and Twitter are two totally different services which can't really be compared. I admit they are not perfectly analog, but my thoughts originate mainly from the fact that these two players will probably be those who'll define how the social web of tomorrow will look, from the point of user experience as well as an organizational and marketing tool. Where Google dominates Web 1.0, Facebook and Twitter are obviously becoming the leaders of Web 2.0, and the fact is any of them (including Google) has a good chance to define Web 3.0.

What's new

The past few weeks have been pretty intense for Facebook and Twitter. The two Web 2.0 rivals both held conferences about how their companies are doing and revealing plans for the future. Facebook's (f8) main presented functionality was the universal Like button, which was already adopted by more than 50.000 websites in its first week, even though with a bit of problems. Twitter (Chirp) announced its service has more users than expected (105m registered, 180m unique users a month) and their main focus in the future will be Twitter Annotations, used for making Tweets embedded with meta-data used for better indexing and search. These two features imply that one of the main battlefields of the future of web will surely be the semantic web, for which experts say at this point Facebook has the advantage.

Real-time web

Leaving aside speculations about where this whole story is headed and how the semantic web will turn out, and concentrate on another thing hot right now: real-time web. Real-time web has so much potential even Google wanted a part of it. Sadly, Buzz came out a bit of a failure, and most of it's traffic is non-generated. The giant probably doesn't really care because it can display Tweets and other streams in it's search results, finding a symbiosis with real-time web services, similar as it has with Wikipedia. But still, it's "new" Buzz service proves that real-time web is interesting for everybody, even the biggest web company in the world.

The main point of real-time web is already hidden in its name. It's about information when it happens. News portals and blogs are minutes if not hours behind, after all, they usually present professional and lectured articles with sources and photographies. But in this hectic and hyper-speed world we live in, information has the highest value when it happens. In the case of extraordinary and unpredictable events, such as natural and man-made disasters, or just casual popular things, such as the premiere of a new movie, timing means everything. That is why journalists turn to these sources more and more, and CNN occasionally analyzes the twittersphere live on television, taking advantage of these new modern media platforms. They actually deserve to be congratulated for seizing this great opportunity.

Privacy issues

Where does Facebook fit into this picture? Well, to be honest, I think it's still trying to fit in it, and that's what the whole "Privacy and real-time web" in this post title is all about. To understand what I'm getting at, we must go a few years back, back to Facebook's beginnings. Facebook started as a closed network for elite schools, then slowly opening up to general public to become one of the biggest websites ever. The service and concept was originally based around privacy – where everything you did you had a full control over who will see it, what probably made it big in the first place. Today, this privacy is almost gone, and even Mark Zuckerberg admitted it doesn't interest them any more.

This decline of Facebook's privacy is concerning many people right now, and you can check out this interesting evolution for yourself (also visualized). Activism and search for alternatives have already made it to the mainstream and it looks like I'm becoming a part of this movement too. Don't get me wrong, I still think Facebook is a great service both for personal and professional life, but sometimes you just have to not want it all (sounds familiar?). It's a bit ironic actually, the thing which made Facebook could also become its end.

In the case of content providing, real-time web and search, for which Facebook perhaps also has ambitions, privacy is a giant barrier. Facebook has content, not only that, it has the most content of them all (including multimedia), but this valuable content is entangled inside Facebook's huge web of (dissolving) privacy, making most of the streams unreachable for general public. Even Facebook Pages, designed to have their information fully open to public, were not helping a lot, because it's the the millions of microbloggers, from the most influential, to the most insignificant, who are empowering the real-time web, and not brands and corporations. Real-time web is a giant crowdsourced news network with reporters scattered all around the world, and that is something Facebook so desperatly wants to be a part of, using all means necessary.

Twitter is the king

We will see if Facebook went too far or will it be able to become a real player in real-time web too. For now, Twitter seems to be the dominant platform and the fact that it's entire timeline will be archived in the Library of Congress is an amazing achievement proving this theory. Facebook's only option to make it on this market is to continue getting rid of privacy, making it's content available for everybody, both human and machine. Of course, there will be other players too and uncle Google will sit right on top of it, trying to use all the platforms and services to its own benefit.

Number of users is the crucial component of real-time web, so Facebook could probably have a greater potential in this field in the long run too. But the truth is Facebook users are becoming annoyed, and it's still a question how they'll accept its new strategic policy and how intimidated they will become by its frequent debacles. But the attachment to Facebook can sometimes be too heavy, so it will be interesting to see if users will actually leave Facebook or simply adapt to its new privacy, continuing to feed it with accessible real-time content.

If the future is semantic web, reality is real-time web, and at this point Twitter is very much in the lead. In the end, it probably doesn't make a lot of difference for a casual user if he gets the information late, but still, the whole concept of real-time web is quite amazing and holds great potential for many people and businesses. An interesting thing, the World Wide Web is, and everything that came with it. Welcome to 2010.

Check out the complete Facebook vs. Twitter series.

How mini-feed and other streams revolutionized IT

Sun, 17 Jan 2010 16:59:20 GMT

Web 2.0 brought us a lot of interesting things, approaches and solutions, so it's hard to imagine how we ever managed without it. Most of us are half addicted to floating around in cyberspace, looking at what people are doing and talking to everybody at once. It is a little social revolution that we are witnessing, and it doesn't seem it will end soon. Web 2.0 changed our everyday lives, but for an IT expert such as myself, these new concepts also have a big influence on other sciences and in our case, they will change management, organization and corporate IT forever.

Besides the whole social networking concept, one feature of Web 2.0 stands our as one of the main building blocks of this new paradigm. Call it the news stream, mini feed, activity log or whatever, it is the little information report that makes you come back every hour to see what other people are doing and if something new happened. Without it, Web 2.0 portals and services would be boring and useless, and they definitely wouldn't be as attractive interactive as they are.

Besides being cool and addictive, the mini feed is also a major shift in the presentation of data and information. Until it came around, data was static, but now, we can notice that data has the ability to become dynamic. In Web 2.0, this feature was "invented" to amuse users, but this approach is also fully applicable to business oriented software. Imagine a corporate solution that is intended for project management, where projects are consisted of tasks that have people working on them. The simplified dashboard for a project would probably look something like the table below.

Two weeks ago	Today
Write the documentation (John - Completed)	Write the documentation (John – Completed)
Delegate tasks (John – Active)	Delegate tasks (Mary – Complete)
Design wireframes (Mary – Active)	Design wireframes (Mary – Active)
Enter data (Unassigned)	Enter data (Simon – Active)
Make HTML (Simon – Active)	Make HTML (Simon – Complete)

The two dashboards, even though they represent the same project two weeks apart, look quite similar, and they actually don't give those, who are not directly connected with the project and people working on it a lot of information about the flow of activities, data and work. This data represented is static. But if we add another layer of mini feed / activity report on the same case, things start looking more interesting and give a lot more information about the project. The data becomes dynamic. When we look at the dashboard above, we don't really know that this is what actually happened with our project.

12 days ago – John changed the status of the task Write the documentation to Active.
10 days ago – John added a new note to the task Write the documentation.
10 days ago – John changes the status of the task Write the documentation to Completed.
8 days ago – John made Mary responsible for the task Delegate tasks.
8 days ago – Mary made Simon responsible for the task Enter data.
6 days ago – Mary added a new task Contact client.
5 days ago – Simon changed the status of the task Make HTML to complete.
3 days ago – Mary deleted the task Contact client.
2 days ago – John added a new file to the task Make HTML.
2 days ago – Mary changed the status of the task Make HTML to Incomplete.
1 day ago – Simon added a new note to the task Make HTML.
Today – Simon changed the status of the task Make HTML to Complete.

This dynamic component wasn’t available in business oriented software until now, except in the most sophisticated analytical systems, where data mining and other high level approaches provided dynamic information. Using Web 2.0 approaches such as the mini feed, operational and transactional information systems finally got the dynamic component too. Not to amuse, but to inform.

These are the approaches we use and things we think about when developing our software solutions. There are actually more useful applications of the mini feed we already noticed, but some mystery must remain. If you are interested, say hi to Neolab, one of the first companies in the world to offer fully integrated Web 2.0 features in business oriented software, all part of the IT 2.0 concept we are developing, for which already got an award for. Made in Slovenia, shipping worldwide.

New features on the chronolog: Blogroll, RSS, Twitter client

Sat, 16 Jan 2010 20:10:00 GMT

Less than a year after launching the chronolog, it’s already obvious that I made the right decision by developing my own platform. Usual blogging platforms are rigid and resemble news portals. I went more towards web 2.0 approach, because I like thinking out of the box and making something special. Hundreds of hours went into chronolog's development and some modules don't actually bring any real value add, but still, I think that the features I built in make it fully comparable with any other blogging platform. Now I have my hands free to take it anywhere I choose and believe me, it will be fun, creative and interesting.

The first wave of features were focused mostly on internal optimization and getting the most out of the data available. The recommendation engine is working like a charm, and the statistics and analytics offer interesting insight into the chronolog information flow. Now, the time has finally come to go towards externalization, towards openness and connectivity with other platforms, services and people. Information must flow not only on the inside, but also on the outside.

RSS

My blog's core is based on RSS feeds, where information from various sources get combined into one unified stream, the chronolog. A lot of other services and aggregators on the market also stand on the basis of RSS. Now, finally, after all this time, the amazingly anticipated, so much appreciated, different RSS feeds for different chronolog views are available for your reading pleasures.

Twitter client

I'm saving the best for the last, and this one is pretty awesome. My chronolog is now a registered Twitter client, able to post links and thoughts directly to Twitter using its API. This is both cool and useful, because until this point, the chronolog was only able to read data from various sources, but now it's also able to generate data. This opens a whole new set of interesting opportunities, both in the direction of building a complete Twitter client, as in the direction of full integration with other online services. Did I mention it was cool?

I'm quite happy with the whole situation and the chronolog's development. Sometimes it's hard to invest both into content and into technology on one single project, but this thing has already gone too far not to use and abuse it even more. I like challenges, so If you have any cool suggestions, they will be more than welcome, but for now this is it. I like to play.

The decline of web forums

Sun, 13 Dec 2009 16:05:53 GMT

The Internet, specially the World Wide Web as we know it today is all about interaction. The first generation of web applications supported little of it. Most of the web was "official" authorial content, but at some point the world was ready for a step forward. User generated content was manifested through forums or discussion boards, which gave surfers a newly discovered access to tons of "unofficial" knowledge. The boom was driven by user interaction and necessity of sharing ideas and thoughts. Looks like times are changing again and forums are dying, at least in the form we knew them. What the hell happened?

An important thing happened, and we call it Web 2.0. The new generation of services supporting networking, cooperation and higher levels of interaction made hierarchically structured knowledge repositories separated from authorial content obsolete. Today, the content became integrated with interaction to form even richer content. I think three types of services that did most of the kill: wikis, blogs and new types of user interaction. I wouldn’t put chats or instant messaging in the same category, because they were around since ever (f.i. IRC) and they leave nothing behind (at least publicly), making them useless for broader crowd and future generations.

Wikis

Wikis are actually some type of a forum, where people gather content and knowledge. Instead of having knowledge scattered around in threads with comments and replies, all the users are working on the same "article", making it better and better as time and knowledge progresses. Few people know that wikis are not just Wikipedia (which is by the way a great example of human interaction achievement), a lot of companies use the same engine to build their knowledge base and web portals use them to build their web presence.

Blogs

With Web 2.0, people went from anonymity behind nicknames to front row participation and ego building. Instead of participating in forums, millions of bloggers started making synthesis of useful forum threads to short and highly informative blog posts with an interesting side effect - building personal brands. Google’s page rank and other search engines did the rest, making forums less important and good blog posts better ranked and easily found.

New types of user interaction

There was a time when the only interactive thing you could do on the web was writing or replying (perhaps vote on a poll which had nothing to do with a content, at least technically). Today, you can post, view, like, dislike, support, comment, vote etc.. This fact gives users more flexibility on how involved they want to get with the content and the content becomes more informational. Knowledge is not hierarchically structured anymore, it’s rather scattered around the web in forms of multimedia (text, audio, video, etc.), with interaction activities attached to it (likes, comments, etc.). That makes it fun and more interesting, and if it’s good enough altogether, it will get synthesized and rebuilt into another form of information.

I think there is still space for forums on the web, but mainly in the functionality of huge chat rooms. On the other hand, general portals will probably be shifting more to combined approach, without the "Forum" link, but with integrated Web 2.0 services and approaches. This doesn’t have to mean using friends and connections, it can easily be the form of better support for different interactions and focus on participation of users around authorial content. Good news for users, bad news for portals.

An approach to statistics and data analysis

Mon, 30 Nov 2009 20:32:06 GMT

When information systems evolve, they become greedier for both operational and advanced strategic statistics and data analysis. This need is a part of a natural evolution. The more data you have, the higher potential for extracting information you have. Looking at business environments using IT platforms, that's what analytics are actually all about - getting useful information from usually bad data. It turns out the task of analytical reporting is not so complex as it seems, but you definitely need a set of different skills / people to make it work.

There are tons of different statistical approaches, methods and theories, but it turns out that for average business needs you only need basic mathematics, where the most complex operations are sometimes logarithms. So, if it's so simple, where does the problem lay? Why do information systems often lack analytical support, which can be used for decision making?

In my opinion there are three main steps to consider when trying to make useful statistics and data analysis, and ignoring or underestimating any one of them will make your reports suck.

Data

Data is the king. If you don't have the data, you might as well give it up. If your data is bad or weak, you might consider rebuilding it. But you should know one thing - the better the structure of your data is, the better your analysis will be. Using a flat database such as a text file or an Excel spreadsheet gives you few analytical opportunities. Relational databases, such as Access, MySQL or SQL offer cross-data querying and advanced reporting, but huge and complex calculations can take a lot of time. For those, a multidimensional OLAP database designed strictly for analysis becomes the only option.

Challenges in this step: Technical

Information

The data discussed above defines the scope of potential information you can deliver. In this step, the main goal is simple - you need to know what you want to know. Business needs, process flow, strategic goals or just plain simple amusement are the main factors that need to be addressed. Having someone who is able to recognize these opportunities is crucial, because data is just numbers, but aggregated data - information - is knowledge. It's quite clear you won't be able to get something if you don't know what you want to get.

Challenges in this step: Analytical

Visualization

A picture can tell a thousand words and this goes a long way for data visualization. Even if you can't use charts, you can color information and use measures such as font size to represent another dimension of information or trends. Besides, always keep in mind that less is more, so you should put irrelevant information in the background and punchlines in the spotlight. Check out different chart types, they're useful for different representations and experimenting with them can display things that don't seem there at first sight. Observe patterns. Try to imagine a playground, where information can satisfy your curiosity and while doing it, it also brings useful and valuable results.

Challenges in this step: Creative

If you have the will, you can do all sorts of crazy stuff with statistics and data analysis, but you should know they sometimes take a lot of time. I'm proud my chronolog already has two nice looking children of these activities. The first one is a simple recommendation engine used for content ranking and the other one a set of reports which offer insight into activity and interactions of the chronolog. What can I say, I like to play around, and it may as well be any information system I can get my hands on. Give me the data and I'll give you information.

Hot on the chronolog - and how it works

Sat, 10 Oct 2009 19:49:41 GMT

When I first published my chronolog, a few people were making remarks about how it resembles FriendFeed, Twitter or Tumblr. I can't deny that. The influences of Web 2.0 are huge both on my personal and business life, so why should the chronolog be any different? It is a mashup of different web services and it displays information from different sources, so it's a kind of a Web 2.0 stream. But besides that, it's also my own personal playground for testing and developing high level services and functionalities, which will hopefully be cool and fun and make the chronolog interesting for all of us. Demonstration of concept and technology, if you like.

I already have a few of those smart features planned, and I can give you a little teaser already. I really look forward to developing the custom view of the chronolog, where advanced users will be able to do a bit of configuration. The prototype is already half developed, but sadly far from production. A different thing I'm working on is a complex set of statistics and analytics, which should give us deeper insight into the chronolog, it's data and our interactions with it. This one will probably go out next and it actually inspired the one already complete. From this day forward, the chronolog supports Hot on the chronolog, accessible from the views menu top right, which shows the most interesting posts in the desired time period.

A few Web 2.0 portals (specially those oriented in social news or social bookmarking) have recommendation engines, which give users access to information based on their interaction with the system. I would like to try that one out too once, but because I don't have registered users, the chronolog probably won't be the environment. What I can give you now, is the Hot view, which displays the most important posts based on the interaction (views, likes, comments) of all users of the chronolog. A global recommendation engine of some sort. I'm quite pleased with the algorithm I've developed, it looks like it's working, so you can give it a try.

Some of you will be interested in how it works. The core is a really super mega awesomely complex algorithm that gives ponders to different interactions in the selected time span. Well, it's not that complex from the mathematical point of view, but it still pretty smart. Combining these ponders and number of interactions, using a few square roots and logarithms, plus a small modifier for insert date (if two posts are tied, the older one appears "stronger"), it calculates which posts are more interesting and relevant and gives them a score accordingly. Simple as that. Besides, is also able to make that calculation for any time period. You can even hack it by changing the ?d=# in the url to any number of days you like.

When viewing longer periods (months, years), blog posts will probably take most of the top spots, because they are supported with social networking sites and have the most interactions. In the shorter periods (days, weeks), other types of posts will also take higher ranks. We will see if the algorithm works in the longer term too, when more users will be clicking around, but if needed, the calculation will be changed or modified. Oh, I almost forgot about the design touch I added - the importance of a post is portrayed using transparency, which looks quite cool and is a great exaple of using design for function.

The chronolog becomes smart. Hope you like it.

Organizing music collections using iTunes

Wed, 02 Sep 2009 20:32:46 GMT

When I got my first iPod a few years ago as a business gift, I was overwhelmed. The thing was pure cuteness and usability. Then I got a cold shower few minutes later, as I tried to put my mp3s on it. The damn thing wouldn't work without iTunes. Apple's strategy to force software to users is a bit Microsofty, and is in my oppinion one of the worst things the company is doing, getting criticism all around. But It turns out this approach is helping them on their world domination tour that's been going on in the past years, as more and more people are switcing to Apple and Macs.

Switching from Winamp to iTunes got me acquainted to music libraries. Winamp also supports it, but it was a bit in the background those days. iTunes had it in the main and at first I though that's just another stupid thing I had to do for nothing. But the more I went into it, the more fond I was of it. Today I am a power iTunes user and I think my library is a piece of art, with houndreds of hours put into it's organization and optimization. Meta data is very powerful, and having songs classified, tagged and labeled leads to simple and flexible use, after the bigger initial investment.

Here are a few approaches that can help you keep my giant library under control:

Labeling and smart playlists: Using correct labels for artists, genres, etc. and rating songs enables the use of smart playlists. I have one main playlists that combines all songs rated more than 2 stars. I have a few similar which focus on specific genres. These are the playlists I use the most. Their biggest advantage is that I can add new songs to iTunes and after I rate them, the automatically go to all the correct playlists, so I don't need to worry about making generic playlists with listenable songs.
Using album covers: This feature is very cool for browsing album collections. Instead of having to read what's in the library, you just look for a specific album cover. They say a picture is worth a 1000 words and it's true. By using cover flow or grid view, you can quickly and easily find the album you are looking for.
Using folders for playlists: Albums should be organized using folders. In my case, I use a root folder named Albums, which contains sub folders according to genres (Rock, Electronica, Pop, etc.), which contain subfolders according to sub-genres (Classic Rock, Alternative Rock, Indie Rock, etc.). This allows me to easily browse (or shuffle) all albums, if I feel more specific, just albums of one genre, and if I am in a really picky mood, just specialized collection of music from a sub-genre. This structure could be achieved by using smart playlist focused on genres, but then all the singles would also be added to the lists, so I prefer managing this by hand.
Using Comment as a tagging tool: Comments can be used as tags, because smart playlists can be configured to play songs where comment contains a word. This means that the system will work even if you put more words (tags) into the comment field. I use comments such as Replace (this song needs a better version), Label (this song is not correctly labeled) and New (this is something new and should be listened to - because date of adding to library is not correct in all the cases) to have a better overview over the songs in my library, and better choice for listening.
Making advanced logical statements: iTunes smart playlists only work with ALL or ANY of the rules or conditions you apply to them, which corresponds to mathematical OR or AND statements. I can set the smart playlist to contain ALL the songs of the genre pop AND rated more than 2 stars AND last more than 2 minutes. Another possibility is to set it in a way it contains ANY the songs that have a bit rate of less thant 192 OR have a comment that contains Replace (these songs need to be replaced). But you can't combine both OR and AND statements in one playlist.
Actually there's a workaround, but you have to put a bit of efford into it and use a folder as a container. All the playlists inside the folder will be treated between themselves as ANY (OR), but each playlist can be configured as ALL. For instance, if you want a playlist of both best and fresh rock songs that will contain (ALL rock songs AND rated at least 4 stars) OR (ALL the rock songs AND made in the year 2009 AND rated with any stars), you just create two separate smart playlists (one for best, one for fresh), add them to a folder and the folder acts as the complex AND/OR playlist, combining both.

These techniques can probably be used also with other music libraries software and can hopefully make easier and better use of music collections. And if you know any other cool tricks, please let me know.