Where's that confounded data?

Where did I put that data again?

I get data in email attachments, in PowerPoint, in Excel, in Crystal Reports.  I read data on the web.   I get data on phone calls, over IM, from the mouths of customers.  I hear about data that other people have and ask, sometimes timidly, for a copy.  Like my brother’s MP3 hording I want to have it, just in case.  There is a comfort in having all of this data.  That is, until I need to find it.

Finding data is hard.  With Draconian email quotas and the difficulties of searching archives, it always becomes lost.  I can find my music now in iTunes, but what about my data?

More and more I use two very different tools to help me survive this information-saturated world: Evernote and SAP BusinessObjects Explorer.  Think of Evernote as your external brain, where you can store your every thought for perpetual retrieval, and Explorer as your company’s external brain, where every last bit of data can be retrieved instantly.  I'll talk more about Evernote later, but this week seems to be all about BusinessObjects Explorer.

Yes, this week SAP BusinessObjects Explorer is being launched, but the technology antecedents to this powerhouse have been around a while.  It is comprised of a product previously known as BusinessObjects Polestar and some cutting-edge in-memory acceleration technology previously manifested in SAP Business Warehouse Accelerator.  Yet it is their combination is truly stunning.

I am an impatient man.  I love my Mac since I open it up and it is ready with no delay – I cannot wait for something to boot.  I have loved using Explorer over the last year since I can hit a web page and answer my questions as soon as I ask them – I don't have to wait for someone to build a report.  I just have a conversation:

You say revenues are bleak for a certain product in the United States?  Let’s look at it by city.  Whoa.  San Francisco and Washington are down, when they are up for the rest of the business.  Let’s get California sales on the phone and see what’s happening – send them the link.  Richard, what’s going on here?  Oh, we didn’t run the right campaign?  Let’s budget for that next quarter, it showed good results everywhere else.

[caption id="attachment_201" align="aligncenter" width="500" caption="Fun with Data"][/caption]

Bingo.  In a 60 minute meeting, we can have 5 conversations like that, and have discussions based on substantive fact.  Coming from the data-desert of past jobs, this kind of knowledge oasis is intoxicating.

Yet combined with the in-memory technology the potential is breathtaking.  Terabytes are now your friend, they are not demons threatening to slow your life to a crawl.  Petabytes are an afternoon snack.  You don’t have to look at statistical samples, you can look at the truth.

So as I index the world around me, play a little for yourself online.  See what it is like to manipulate 1000 rows of Excel in this remarkable tool.  And then imagine what it would be like to manipulate the world.  That is, if you can handle the truth.

Torture is bad, everywhere.

In Torture, Plain and Simple, Suzanne points out that while it is true that torture doesn't work, it doesn't matter: it's illegal.  But let's dwell for a minute on the first point: torture doesn't work.

Elaine Scarry gave this subject a scholar's attention in The Body in Pain, where she explained that pain nullifies the world around us -- with extreme pain nothing exists but the pain.  This deconstructs the ego to a point where conversation is meaningless and information extracted in this state has one goal: to make the pain stop.  Say anything to make the pain stop.  In fact, there is a long history of torture being used to extract misinformation to support campaigns of misinformation.

While this simple fact is well established in research, it seems appallingly under communicated.  If it was well communicated, I imagine it would lead to this:

Interrogator 1: Should we do it?

Interrogator 2: Well, it doesn't work.

Interrogator 1: OK then, let's not bother.

The complex ethics simply disappear.

The Art of Trust

I was about to tweet about some happy cows I saw while driving to work today, until I remembered the law and thought better of it.  The cows weren't worth a ticket.  But aside from the desire to avoid tickets and stay alive, there is another hazard of automotive texting - thumbing the wrong key and sending the wrong message, perhaps to the wrong person.

When I hired a guy last year entirely over SMS I committed a gaffe - I received a Twitter DM (direct message) and hit reply, sending the reply to all of my followers.  This is a variant of a DM Fail, when people think they are sending a message to just one person but instead broadcast it widely.  In my case I uttered something relatively harmless like "req opened this week."

Something more nefarious happened recently on Twitter: they actually sent DMs to the wrong people, detailed in TechCrunch.  Jason accurately called this a "breach of user trust," but it was resolved quickly.  Twitter is not alone here.  A colleague of mine was using an esteemed Web2 product when they one day got a trove of someone elses messages dumped on their desktop over IMAP.  Only once, but once is all it takes.