crunching big numbers can help us learn a lot about ourselves. ..... But no matter how big that data is or what insights we glean from it, it is still just a snapshot: a moment in time. ..... as beautiful as a snapshot is, how much richer is a moving picture, one that allows us to see how processes and interactions unfold over time? ..... many of the thi
Structure of Evolutionary Biology - Blue (Photo credit: Wikipedia)
ngs that affect us today and will affect us tomorrow have changed slowly over time ...... Datasets of long timescales not only help us understand how the world is changing, but how we, as humans, are changing it — without this awareness, we fall victim to shifting baseline syndrome. This is the tendency to shift our “baseline,” or what is considered “normal” — blinding us to shifts that occur across generations (since the generation we are born into is taken to be the norm). ..... Shifting baselines have been cited, for example, as the reason why cod vanished off the coast of the Newfoundland: overfishing fishermen failed to see the slow, multi-generational loss of cod since the population decrease was too slow to notice in isolation. ..... Fields such as geology and astronomy or evolutionary biology — where data spans millions of years — rely on long timescales to explain the world today. History itself is being given the long data treatment, with scientists attempting to use a quantitative framework to understand social processes through cliodynamics, as part of digital history. Examples range from understanding the lifespans of empires (does the U.S. as an “empire” have a time limit that policy makers should be aware of?) to mathematical equations of how religions spread (it’s not that different from how non-religious ideas spread today). ...... building a clock that can last 10,000 years .... the 26,000-year cycle for the precession of equinoxes ...... Just as big data scientists require skills and tools like Hadoop, long data scientists will need special skillsets. Statistics are essential, but so are subtle, even seemingly arbitrary pieces of knowledge such as how our calendar has changed over time
I just received my third email in as many weeks asking me to show up for Sam Lessin's sci-fiMeetUp called Y+30. That is not spamming, that is bombardment. I was sold with the very first email though, even though I was not as impressed with the last event they had, the one on food. Most speakers at that event tried to venture out about five years at best; or I was not able to follow much of what they said. And food as a topic does not excite me. When I think food, I think hunger. I am a Third World guy. I consider myself a great cook, but I cook a very limited number of items. I have never used any recipe or cookbook for anything. The whole recipe talk is designed to make believe anyone can cook anything. I don't buy into that. On food my thought is I like to stay thin.
The next MeetUp in queue is called The Future Of Porn. SamLessin, visionary? Or a visual guy? There is a clip in the movie Minority Report that he might want to show at that one.
But the one tonight is called The Future Of Databases. Now this is really something.
Thursday, May 20, 2010 7:00 PM Drop.io World HQ
68 Jay Street Suite # 413
Brooklyn NY 11201
Like most things, the long-term future of databases is best discussed over beer... With the help of our friends at Basho Technologies we've assembled a cutting edge panel of experts to tackle what databases will look like +30 years.
Our panelists represent the full range of high volume storage engines from the document store avant-garde to the RDBMS old guard and everything in between. You can expect thorough coverage of what "database" means today and what it will mean in the future.
Panelists:
- Ken Ross - Professor in the Computer Science Department at Columbia University
- Dwight Merriman - CEO, 10Gen (commercial sponsors of MongoDB)
- Jonathan Blessing - Managing Partner, 3Thirds Software (NYC-based DB consultancy)
- Justin Sheehy - CTO, Basho Technologies (commercial sponsors of Riak)
In case you have not visited the site recently, Drop.io has gone through a major revamp. The main page is much more minimalist now. That is a good thing. Don't make people scroll. Every milli second counts. Fast is good.
This is going to be my third time at the Dropio offices. The first time was during Social Media Week early in February, the second time was only a few weeks back at a Digital Dumbo party. Two days before that I got to meet Dropio Dude Jacob Robbins for the first time. At the party Jacob told me about the new look of Dropio. In this blog post - Digital Dumbo: Here I Come - I talk about Jacob and something hack. All Jacob ever did was he spoke to me the web address for the page that tells you how to get to the Dropio offices, fancifully called the Dropio World Headquarters.
Having this event at the Dropio offices is a smart move on the part of Lessin. Dropio is one of the top 20 dot coms in town. It is a startup that has crossed that threshold of whether it will survive and do well or what. But even FourSquare, the dot com in town with the most buzz, is scheduled to lose most of its buzz by the year end when it will be in a much better shape as a company - measured by the fundamentals - than it is today. But it is good for team morale to have some buzz. And hosting a get together like this one gives you some buzz in the NY tech ecosystem. And it is not just Dropio, but Dumbo, that unique locale like none other in the city when it comes to tech. The mystique is fast building.
Once you have figured out that there are four screens - the phone, the browser, the desktop, the wall - it is easy to see search is pregnant territory, rich with possibilities. Search, content, distribution. There is so much to do.
Search is of special interest to me. There is one clear leader, but the challenger is not lacking in money. And for the leader the competition is but one click away. If there was ever an innovation challenge, this is it. Could Microsoft innovate, or find a startup or two that might? That second possibility is more likely. But then Google has been in competition with itself. It never stopped innovating in the search domain and boasts a larger index than anyone else. And its algorithms are still the ones to beat.
Microsoft should try and eat into Google territory. Google should try and eat into Microsoft territory. The consumer will benefit. Search and operating systems are fair game.
How Facebook Copes with 300 Million Users Technology Review Doing things in or near real time puts a lot of pressure on the system because the live-ness or freshness of the data requires you to query more in real time. ...... There's too much data updated too fast to stick it in a big central database. That doesn't work. So we have to separate it out, split it out, to thousands of databases, and then be able to query those databases at high speed. ...... "Like" became one of the most common actions in the system. ........ a tremendous wealth of photos being uploaded and shared ......... Then we went and built our own storage system called Haystack that's completely built on top of commodity hardware. Twitter's Growing Pains a large-scale, ground-up architectural revamp ...... it will reduce its reliance on Ruby on Rails, and will move to a "simple, elegant file-system-based approach," to replace its original unwieldy database system. ........ a communications-class technical infrastructure that supports unpredictable activity. Social Networking Is Not a Business How Facebook Works
![Reblog this post [with Zemanta]](https://lh3.googleusercontent.com/blogger_img_proxy/AEn0k_sIuw5xuJyAswCaiwDPwQmv1NzkKvORRFNkkXwiYUYEzYOrXGnnoK5n6yfkgG5u9JupuzAqlMPMyQBLmhJozNxkBG45iLLq0gcXCdMiKile3T1WXOffdpkD8PoN5uOksmigX9XCPx99XVcJr4NbN4Hf=s0-d)
![Reblog this post [with Zemanta]](https://lh3.googleusercontent.com/blogger_img_proxy/AEn0k_v1NNCM6Smg8wKQW3PmiJaVFQ-RuRiDZl8Isg1OjHu5eU5qd6F0i9EBEdS4M4dh19i_1xD4fFUd7tws9-Z1IzHGgUID07QcijnS99lrTVZoq3X0_MFW3gsOkS5LnZ5YUt90q3uuaJ1Roo7x29CUHoD7=s0-d)