Panorama photo created by Microsoft’s Windows Live Photo Gallery. For details, read on.
In two earlier dispatches, here and here, I suggested that you give Google Photos a try if you hadn’t done so already. The most obvious payoff is providing an answer to the increasingly pressing question of how to handle increasing zillions of digital images. The less obvious advantage that has grown on me involves the big data/AI aspects of the system — the way it automatically groups photos of one scene into panorama views, or created animated GIFs, or creates albums with titles like “Iron Pigs Game in Allentown” by recognizing the landmarks and activities.
Yes, I know that on the other side of this same technology is the Panopticon Surveillance State. For now I’m talking about the applications that many users will find convenient — and, in particular, where they originated.
A reader who was a long-time official at Microsoft writes in to lament the fact that Google is now being either oohed and aahed at, or viewed with concern, for innovations that Microsoft had been chipping away at for years, especially with a product called Photosynth. I turn it over to the reader:
Once again I shake my head at the ways my ex-employer [Microsoft] would develop a great idea, and then not follow through, partly because of an inability to catch on with the cool crowd.
Photosynth could not just stitch many photos together, it could (given enough pictures) create a 3D model – about 10 years ago. It’s not as passive as Google Photos, as you have to point it at your picture collection, but it would have been relatively easy to make it munch on photos uploaded to OneDrive. The idea has basically just sat as a research project, with the last update 9 months ago, and now Google gets the credit for a great innovation. Apparently people are still using it (who knew?): go to photosynth.net for some recent 3D creations.
I’d never known of this project; I went to the site after getting his note; and he’s right! It looks very interesting. For instance, check this out. Back to the reader:
Apple didn’t invent the smartphone. The “Pocket PC” phone came years earlier. Nest? I remember touring the demonstration e-home when I joined Microsoft in the late 1990s. [JF note: when I was at Microsoft in 1999 I saw it too.]
Now, maybe the company didn’t pursue these projects because they weren’t part of the core mission, and maybe it was because Ballmer was focused on business and not “toys”. That would be fair rationale. But it is, at best, ironic that Microsoft gets the reputation of never innovating anything, and being a ruthless productizing and marketing machine, when almost the opposite is true….
Re your correspondent’s concern about big data and privacy: that cat has been out of the bag for a long time. The “cloud” being able to figure where you were and when dates to when the first geo-encoded picture was uploaded to flickr.
After further comparison of what Microsoft had done and Google is now doing, he wrote once more:
I guess it’s the [Microsoft Live] Photo Gallery feature that’s most like the Google feature. Photosynth is a world beyond both, and has more of the “wow” factor.
The photos [at the beginning and end of this post] come from the London Natural History Museum and the top of the Arc de Triomphe in 2012; I just had to select the pictures in Photo Gallery and click.
What Google seems to bring to the table is (a) identification of a group by place and time (which is trivial), (b) recognition of picture similarity (which is commonplace today) and (c) lots of computers available to make panoramas in the background and probably (d) recognition of a good result. And I guess they trim the result to a rectangle.
The fact you of all people hadn’t heard of Photosynth is exactly my point.
Photo Gallery panorama of Natural History Museum in London
Recently I recommended that you check out Google Photos if you have not done so already. Like Gmail, it’s a way to store huge quantities of digital material and leave its management to someone else. (I promise, later we’ll get into the privacy tradeoffs involved.) And much more than Gmail, it offers big-data tools that can arrange and transform your information/photos in ways difficult or impossible to do by yourself.
For instance: I mentioned that Google Photos had, on its own, merged three smart-phone snapshots of a scene at Oxford into one panorama view. Several people wrote in to say: Let’s see the originals! So here goes.
First, in its full-frame entirety, a smartphone snapshot of one side of the entry quad at The Queen’s College, Oxford.
Then two almost-identical shots of the other side, both in full frame. First this:
Then this:
The point is that without my doing anything more than saving all three shots to a Google drive, the system recognized them as overlapping parts of a whole and stitched them together into a high-rez, level-horizon, panorama version, looking like this (and at larger scale here):
Even when zooming in on the composite shot as far as possible, I still can’t find a pixellated boundary where the shots were brought together.
We all say in our blase way: Yeah yeah there’s increasing power of big-data systems. At least for me, seeing how it worked on my own information dramatized these effects. To be clear, this was with three quick, casual phone-shots taken over a few-second span. The result isn’t anything fancy, but it’s different from what I could have done myself.
And, as I say, we’ll get to the surveillance-state ramifications soon.
***
Actually, why not now. Here’s one reader response:
In the vein of the glass being neither half-full nor half-empty, but having a leak, that [big-data] magic means just as much that a computer can figure-out where you have been and when based on the photos you take, making it that much easier for a human being with access to that computer to know where you have gone and when.
The photo taker providing in essence CCTV of their movements. At least though unlike CCTV (and of course all that magical facial recognition) if the photo taker stops taking/posting the photos the intelligence stream stops.
Front courtyard of The Queen’s College, Oxford, recently, in a panorama that Google Photos automatically created from several overlapping smart-phone shots. Here is a link to the same composite picture in full rez.
Two quick updates on themes I’ve mentioned over the years.
Hit: Google Photos. You may well already have started using this service — some 100 million people have done so since its debut early this year. If you haven’t, by all means check it out. It is the closest thing I’ve had to the feeling of magic in online life in a very long while.
This review a few months ago, by Casey Newton in The Verge, gives you the main idea. The title, “How Google solved our photo backup nightmare,” covers one main feature. Just as Gmail long ago became the place where it was easiest and most efficient to store, arrange, retrieve, and otherwise handle electronic messages, Google Photos finally seem as if it can be the answer for the ever-mounting volume of digital images. Yes, I’m aware that Google is making use of the vast raw data users entrust to it. Newton’s piece, and another in Vergeby Ryan Gantz, explain why they think (as I do) that the tradeoff is worthwhile.
Beyond the storage-dump aspect is the application of big-data in ways that are sometimes creepy but more often useful and even astonishing. This past summer I took a few camera-phone snapshots at The Queen’s College in Oxford, where my wife and I were married long ago. The next time I logged into Google Photos, it had, unbidden, aligned and assembled the patches into the composite panorama you see above, or here. Pictures you take in the modern geo-tagging age it can of course match to locations. But based on images alone it has gone through and grouped old photos by location — giving me, for instance, a collection of pictures taken in Duluth, Minnesota in 2002, or another from Shanghai a few years later.
As Gantz says:
The service delights by offering me presents. As photos upload, Google Photos is processing old pictures I’ve forgotten about, including images that I’ve assumed were unremarkable or superfluous, and assembling them into collages, animations, and experiences that I wasn’t aware I wanted. “Assistant” offers me its creations and politely asks if I want to dismiss them or add them to my library. Like an opening of Timehop, these little creations can be surprising and lovely.
It’s hard to appreciate this feature until you experience it. I keep eagerly checking Google Photos notifications on my phone, excited about what Assistant has crafted from my digital trail. I find animations of my children playing on the grass, a collage of my wife giggling, a trip to Austin rendered as a slide show.
Let me emphasize the “hard to appreciate until you’ve seen it” point. For instance, here is a GIF animation of a visit to the Southern Tier brewery near Chautauqua in August, which Google Photos auto-created from a set of phone snapshots.
Judge for yourself, but certainly give it a try.
***
Miss: Livescribe Pen. I’ll try to make this concise, because I’m writing to amend the record rather than to beat up on anyone.
Starting six years ago, I have in this space frequently sung the praises of the Livescribe pen. When it appeared, Livescribe was another seemingly magical step forward: it matched notes you made in a special notebook, with audio recordings it was making at the same time. Later on, you could simply click on the notes you’d made — during an interview, at a lecture, in a language lesson — and hear that exact part of the recording played back.
The original Livescribe model, back when everything seemed possible.
The system indeed worked like magic — when it worked. But over the years, I have come to mistrust successive Livescribe models because in the real world, for me, they simply failed too often.
A pen would suddenly and unobtrusively turn itself off during an interview, so that when it was over I saw that I had captured the first 10 minutes of discussion but not the next hour. The first time this happened, I thought it was bad luck. By the fourth time, I’d lost faith. Other sessions recorded all the way through — but then proved to be corrupted and unreadable. With the plain old cheapo Olympus and Sony digital recorders I’d used before, I lacked the fancy features but had never lost information. After another data loss about a year ago, I (regretfully) stopped using Livescribe and switched back to the humble pocket recorders.
My friends at Livescribe tell me that my problems represent an unfortunate outlier experience. Maybe so. But many of my journalist friends say that they’ve had problems like mine.
The Livescribe company was recently taken over by a Swedish firm. I wish everyone there the best, and I wish for a reliable version of this pen. If you’re using one, especially if you tried it on my suggestion, I hope that it’s holding up well for you. But having repeatedly gone on record saying that I used it, I wanted to close the loop by explaining why I don’t any more.
