Monday, April 12, 2010

CAP ACID noSQL


Probably due to the predominant position of a few actors (Google, Amazon, Facebook, etc.), I often feel that all I used to learn about small size systems will soon be obsolete.

The current fashionable IT word is noSQL. If you don't have a minimum of IT culture and only read the buzzy articles you could think the SQL thing is dead.
After a short analysis, I collected a few answers and mainly understood, both old and new technologies will have a long life, but their opposition really makes us step back and reconsider.

First, there is ACID for atomicity, consistency, isolation and durability, the properties implemented by the Database management system (DBMS) to make sure the transactions proceed reliably.

Most of the time we use databases of this type in our indoor computing system. But when it comes to distributed ones (many nodes) you've got to understand that a choice has to be made. In 2000, Eric Brewer made a keynote speech at the ACM Symposium on the Principles of Distributed Computing (PODC) about key principles in the name of CAP:
  • Consistency (all nodes see the same data at the same time)
  • Availability (node failures do not prevent survivors from continuing to operate)
  • Partition Tolerance (the system continues to operate despite arbitrary message loss)
I'm not going to explain what others have done greatly before so read
Sources:
Wikipedia : ACID
Wikipedia : Database management system
Nahurst : Visual Guide to NoSQL Systems
Kennygorman : Dropping ACID
Julianbrowne : Brewer's CAP Theorem
Highscalability : Drop ACID and Think About Data
Blip : Drop ACID and think about data

Tuesday, April 6, 2010

GWT Quake in the browser

2010, the browser is a platform. Step by step, HTML5 arrives.

Three Googlers present their 20% project: a port of the Quake II engine to HTML5 using the Google Web Toolkit.
For more information, please visit Quake2-gwt-port.



In the projet page, you read:

In the port, we use WebGL, the Canvas API, HTML 5 elements, the local storage API, and WebSockets to demonstrate the possibilities of pure web applications in modern browsers such as Safari and Chrome.

The port is based on the Jake2 project, compiled to Javascript using the Google Web Toolkit (GWT). Jake 2 is a Java port of the original Quake II source code, which was open sourced by id software.

To make the Jake 2 code work with GWT, we have

  • Created a new WebGL based renderer
  • Ported the network layer for multiplayer games from UDP to the WebSocket API
  • Made all resource loading calls asynchronous
  • Created a GWT implementation of Java nio buffers based on WebGL arrays (to be ported to ECMAScript Typed Arrays)
  • Implemented a simple file system emulation for saving games and preferences using the Web Storage API
For more details about HTML5, see this presentation.

Friday, April 2, 2010

Toward public cloud

We are now living in the second decade of the 21st century and having data/services online is not an advantage anymore, but not having it is a disavantage. We used to deploy servers in data centers to make sure our customers have permanent access to our services or to give the sensation that we were always online.

Our internal IT infrastructure was changing step by step and the recurrent question used to be: should we based everything around Microsoft to minimise the products' compatibility or should we take the best of every product and support their interaction ourselves?
We were living with the philosophy of IT perimeters delimited by security devices even if users were spending their time collaborating with others, sharing their documents by emails located somewhere. Somewhere connected.

Today, everything looks different due to our understanding of news trends and the near future:
  • the soon to be soaring energy price,
  • the unavoidable trend of collaboration pushed by social networks and the behaviour of generation Y,
  • the old distribution system, like emails, are dying and people communicate on their social tool board,
  • the distributed models of data storage related to the end user's needs.
Consequently,
  • while the cost of hardware is constantly droping, optimising the infrastructure operational cost is a mandatory survival concern,
  • while de-perimeterisation has not been oriented, at least the value of data is now understood, identity and access control are coming next,
  • while the data storage reliability is transparently managed by multi nodes systems, traditional ACID databases have new competitors.
What is the trend regarding system management?
  • Computing resources never get used in small companies : 10-20%,
  • It's only by using huge data centres that the costs will be minimised,
  • Using optimisation by mechanisms like auctioning will be really efficient.
Resource optimisation:
  • Private cloud (belonging to one company) cannot be a scalable solution and enterprise data will move to the public one. It does not mean the enterprise's data is public but the infrastructures are shared among many companies.
Localisation:
  • Cloud regionalisation could be one solution: OK, I may be biased due to my current professional security activity. But feeling that at least a part of my data is not far from home could be less a subject of fear.
  • Also, this is the only solution for software, hardware vendors and IT staff to stay alive, to justify their value.
Now, another huge issue is the way to manage data: so let's talk next about the Nosql movement.

Sources :
Perspectives: Perspectives - Using a Market Economy
Informationweek: Private Clouds Are A Fix, Not The Future
Elasticvapor: ElasticVapor :Life in the Cloud: Amazon EC2's Greatest Threat is Cloud Regionalization