Being a Consultant

Everything I know about consulting can be summed up in the following:

1. Listen to your customers.
2. Understand your customers.
3. Be their trusted advisor.

Listen to your customers. A consultant’s #1 skill is listening. If you can’t listen to your customers explain their problems and needs, you won’t get anywhere. I’ve worked with others who, because they’re consultants, think they already know what the customer needs before the customer has a chance to describe it. It is, after all, the consultant’s job to know what to do; that’s why people hire us in the first place. But this approach is wrong-headed for a couple of reasons. First, “what to do” is highly dependent on what the problem is. Without listening to what makes this customer special, it’s hard to say off the bat what’s right for their situation. Second, listening to the customer gives them confidence in you. They see that you take them and their problems seriously. And finally, often people just really like to get their problems off their chest. When you give them a friendly ear, they unload, and they feel relieved already just because somebody is listening to them, even if you haven’t done anything yet.

Understand your customers. So you have to be a great listener. But the next step is taking what you hear and figuring out what to do about it. This is where the “consultant” part comes into play — your special expertise, the reason people hire you instead of doing it themselves. You need to understand their problems as if they were your own and apply your expertise to give them workable solutions. And by “solution”, I mean the whole deal — not just a tool that fixes their problem or a process sketched out on a piece of paper. You have to consider the whole kit and kaboodle, including the hurdles your customer may need to overcome with their bosses or colleages to actually do something with the tools and processes you’re suggestion. The “solution” should be a way to actually solve their problem, taking into account everything you’ve learned about their situation.

Then, when you have solutions, you have to deliver them in a way that your customer understands. You need to be patient, because they do not have your expertise (remember, that’s why they hired you). You need to be diplomatic, because often your solution will involve telling them they weren’t doing something in the best possible way before, and they could do better. Of course they do want to do better, but they don’t want to be embarassed or feel scolded in the process. This is what we used to refer to as the “your baby is ugly” problem. If someone’s put a lot of time and work into a project — their “baby” — and then the consultant comes along and tells them how awful it is, they’re really put off. You have to be gentle and work with them, and focus on the improvements they can make rather than the mistakes they already did.

Be their trusted advisor. Through this process, your customer comes to trust you. They rely on you. They call you up or have you sit in meetings to get your opinion about this thing and that thing, and they bring you in because they’re starting Phase II and they want your input. This is the kind of relationship you have to strive for — being your customer’s trusted advisor. (I’ve never read The Trusted Advisor, but it’s come to me with high recommendations. I’m adding it to my reading list.) This kind of relationship smoothes over the sales process, because you don’t have to prove yourself to the customer. They know you and they like working with you, and they trust your abilities in guiding them to do the right things. Moreover, they trust you to treat them with regard, to handle their problems delicately, and to not embarrass them with their boss.

That’s it; that’s all there is. Consulting is all about relationships, and if you have good ones, you almost can’t fail. You can be a top expert in your field, but if you can’t do these three things, clients won’t like working with you because even though you’re smart, you don’t feel very helpful. I’ve seen consultants who were really good at listening to customers, understanding them, and being trusted advisors — and even though not all of them necessarily had top-notch expertise on tools or technologies or whatever the client actually needed help with — those people get repeat business, because clients actually feel that they are getting helped.

Of course, to be the best consultant, you really do have to have the knowledge that the client needs. That’s the second part of what I said.

JW’s Philosophy of Learning

Some people are constantly amazed at what I know about various subjects, especially technology and how various pieces of software work. I am going to let you in on a little secret: there is no secret to amassing this knowledge.

(With credit to Dorothea for the phrasing): The way to learn is to beat on things with rocks.

That’s it. Just keep trying until it does what you want it to do. When you get there (or even if you don’t), I guarantee you will have learned something.

OK, I can recognize that this is a statement of philosophy, but as an actual game plan, it may not be all that helpful. So here are five tips on how to be a more effective rock-beater:

1. Define the problem well
2. RTFM
3. Change one thing at a time
4. Someone else already knows
5. Don’t believe it until you see it

Define the problem well. This may seem like a no-brainer, but if you don’t know what it is you’re trying to do, it’s hard to come up with something effective to do about it. Before you start hammering away willy-nilly, take a step back and say, “What do I need to accomplish here? What’s vital in what I’m trying to do, and what’s just nice-to-have? Is this really a problem, or is it caused by something upstream that’s really what I need to be working on?”

RTFM. Read the *#$%^@ manual. (This is especially dear to me because I used to be a technical writer.) Now, I know, not everything has a lovely manual like the kind I used to write. But by gosh and by golly, you won’t know until you take a look, will you? I can’t count the number of times someone has asked me a question that’s perfectly good, but also perfectly easy to answer if they’d spend 5 minutes taking a look at the documentation.

Change one thing at a time. If you’re trying to suss out how something works, you must think of yourself as a scientist performing a little experiment. You must change just one thing at a time to see what happens. If you change two or three or twenty-seven things at a time, how will you know which one is the one that actually worked?

Somebody else already knows. The chances are high that there is someone out there who can answer your questions. Maybe it’s your colleague in the cubicle next door, or maybe it’s somebody with a blog or a book or on a forum or mailing list. (Part of the trick is knowing where to look; do some research. The Internet is your friend, and so is your library.)

If you ask, these people are often eager to teach you what they know. However, a word of warning — there’s a reason this is the fourth tip and not the first one. “Define your problem” and “RTFM” first, or people are likely to be a little peeved at you for dumping your ill-defined problem on them, or asking things you could easily have found out yourself. Get as far as you can on your own, then ask for help.

Don’t believe it until you see it. This is the flipside of “RTFM” and “Somebody already knows”. Some document or person can describe how something is supposed to work, but you should still try it out. (This is especially true when the document happens to be a piece of marketing literature for software.) By trying it yourself, you make it yours. You really understand how and why (or even if) it works.

Good luck

So that’s all I know, and it works for me. I can’t promise any more than that, but I do hope it helped my former coworker, and I hope it helps somebody out there in the blogosphere, too.

I realize this is old news now, but LibraryThing announced an API for work data. This is great. But what’s really awesome? This little tidbit from the post:

Scope. This is an API to work information. Once I’ve worked through the kinks here, I plan to release a member API, allowing members to do clever things with their data. For example, members will be able to make their own widgets, not just rely on ours.

I will squeal with glee the day there is a member API. I’ve been harping on the issue for ages, because I really want a way to make a “to-read” list that mashes up LibraryThing with my local library data. I can’t wait.

Lots of changes have happened. Most notably, I’ve changed jobs. Formerly doing information architecture and content management consulting, I’ve sidestepped fields slightly into web analytics at another consulting firm, LunaMetrics, also here in Pittsburgh.

Web analytics is all about measuring what people are actually doing on websites. You can’t create a good site without relying on information architecture best practices (my old role), but you also don’t really know a site is working as well as it could or should without measurement and experimentation (my new one). My role is again a mix of technical and business knowledge. I have to be able to tweak JavaScript tracking code, but I also need to be able to elicit the goals of a website and interpret the analytics to understand what that means to the business behind it. And, I get to play with numbers, which is fun (I was a math major, after all).

So far, I’m very excited. I like learning new stuff, so I’ve been immersed in web analytics blogs and books, and learning Google Analytics (GA), since we’re a GA Authorized Consultant. Last week we did a one-day training in NYC for about 70 people, where I got to speak about creating a data-driven culture at your company. (The challenges of changing company cultures I am all too familiar with from my previous work.)

So I’m sure you’ll hear more on this blog about web analytics. I also am far, far behind on a post I started writing ages ago about mashups, which I’ll be finishing soon, and I have a handful of other things I’m itching to write about too. It’s been a long dry spell of blogging, but I’m back. Hmm… this place could probably use a redesign, too…

Unfortunately I didn’t get to stay for the Web 2.0 and Open Source Summits, which were taking place Wednesday–Friday. I’m especially bummed because David Weinberger was speaking, and I’m a total DW fanboy. (I squealed with joy when I won his book in the LibraryThing contest giving them away.)

Much of the conference dealt with employing Web 2.0 technologies in the enterprise and dealing with the fact that the work/life line is blurry and users’ expectations for corporate portals, intranets, etc. are set by their dealings with the Web at large (an idea known as “consumerism”). I can certainly relate — I know I expect (sometimes impatiently) that applications at work, from email right on up the chain, work as well as those I’m used to at home.

I found much of the conference confirmatory of trends I already recognize, but there was one thing in particular that got me thinking. Gartner analyst David Gootzit gave a presentation about the future of the portal market. He argued that consumerization will lead to the development of a “portal fabric” for the aggregation of experiences across the portals people use (e.g., iGoogle, your bank portal, your work portal, etc.) — the “Follow Me Portal” or “MyPortal”. The emergence of this portal fabric requires standardization of a number of different functions, such as identity management, personalization and preferences, portlets, and metadata. (Gootzit also argues that this trend is likely to result in enterprise portals being decomposed into component services, something that we’re already beginning to see to some degree with, e.g. search.)

It certainly would be cool if someday My Yahoo! or iGoogle or something else could be your real, honest-to-goodness personal homepage that aggregated all the things you were interested in. Not just your horoscope and the weather and some RSS feeds, but also your bank balance, what’s going on at work, your home automation portal, and so on. (Now, I certainly know there are privacy/trust issues with, e.g. letting Yahoo! or Google access your bank balance, but let’s assume the portal provider is an entity you trust.)

I want to skip over, for the time being, the question of what sort of software the “Follow Me Portal” actually is — whether it’s from a major web provider like Yahoo or Google, or whether it’s built on enterprise portal frameworks within businesses, or by Web 2.0 startups, or even as plugins or customizations to desktop software such as browser extensions or something like Flock. Instead, I want to look at the idea of the “portal fabric” that would be needed to support it. What standards currently exist for federating the functions of portals and where are there gaps? Here’s my still-processing-the-thoughts list…

Identity management

For authentication, we have OpenID. Although it’s not entirely clear yet if OpenID is the winner here, it’s looking better all the time. Big services like AOL and Livejournal are both OpenID providers (and a third party provides OpenIDs for Yahoo! accounts using Yahoo!’s API) — meaning about there are about 120 million OpenIDs out there already, whether they’re being used yet or not. Fewer sites accept OpenID for authentication, but the number appears to be steadily growing — I’m using OpenID to sign into 37signals applications, 43folders just announced they’ll be supporting it, and the other day when I got a trial account to myExperiment.org it asked me to sign up with an OpenID. (You can find more site accepting OpenIDs at myopenid.net.)

For other information about identity, there’s XFN and FOAF. This is especially timely given Six Apart’s David Recordon’s announcement of tools for “opening the social graph”, which is not only about managing your own identity, but also your relationships to others.

There’s also vCard/hCard for directory-listing type info about people. (UPDATE: And duh, I forgot about LDAP.)

Portlets (or widgets, or gadgets, or what have you)

Well, there’s JSR168 and WSRP (and forthcoming updates in JSR286, and WSRPv2) but those are really only adopted by commercial enterprise portal frameworks. Google, Yahoo, etc., aren’t supporting them. Maybe they should, or maybe there’s something else. Certainly RSS and Atom represent really lightweight ways of passing data to a portlet/widget/gadget, but they’re not nearly as broadly encompassing as JSR168 or WSRP are, and can’t fully encapsulate the definition of a portlet to make it portable across these portals. How great would it be if your Yahoo widgets, Google gadgets, Apple Dashboard widgets, etc. were all interoperable and you didn’t have to worry about which platform any particular widget was made for?

I don’t really know a ton about the details of JSR168 or WSRP. I’m not sure whether they represent a viable way(s) forward, or a new, more flexible standard is needed in this category.

Personalization and Preferences

This category is possibly the most tricky to deal with, which is probably why there are few existing standards in this realm. I think there’s also a great deal of value to be gained here, however.

One standard that does come to mind is P3P for privacy preferences. It’s been around for some time but hasn’t really gotten a great deal of traction, although there are some browser plugins and so forth.

Search

Of course, there’s Z39.50, but I don’t know of anyone in their right mind who’s not running a library catalog with a Z39.50 interface. (UPDATE: I forgot to mention SRU and CQL, which are based on Z39.50 but updated for the Web. I think I used to know more about these, but now remember approximately nil. I have to read up again…)

OpenSearch is a more modern, digestible alternative. It hasn’t been around very long, but it’s gaining support both by search engines, wikis, blogs, and other tools (as providers) and browsers (as consumers). And there are extensions to handle more complex searches, geographic searches, and other more complicated things.

Also related: metadata standards. There are lots of these for specialized purposes, but the simplest are the likeliest to be useful for syndication and aggregation purposes. Specifically I’m thinking of Dublin Core here. Certainly DC isn’t complex enough to handle most metadata needs for even mildly complex cases, but what would be interesting is if metadata schemes had a defined reduction algorithm to simple Dublin Core, so that a standard set of metadata could be used by, e.g. OpenSearch. (Search APIs could still support native metadata schemas as well, but I think there’s value in a standard interface for straightforward parameterized searches based on things like dates and authors.)

Publishing

For pushing content elsewhere, there’s the in-development Atom Publishing Protocol (APP), which is mostly thought of as an API for posting to blogs. But wouldn’t it be cool if you could also use it to comment on someone else’s blog, or participate in a discussion forum, or change a wiki article — or even, say, post a link to a bookmarking service?

OK, maybe that last one can stick with specialized APIs. I’m not sure there’s a strong case for, e.g. del.icio.us, to support APP. But for blog posts, comments, discussions, wikis — which are all essentially similar things, just updating some content — it would be killer to use the same interface for them all, right?

Closing thoughts

These thoughts aren’t fully formed at this point, and this is something I’m going to continue thinking about. If you think there are existing standards that I’m overlooking in any of these areas, or if you think there are any areas to be standardized that I missed completely, I’d be interested in hearing about them.

One thing that struck me as interesting is a technical detail. On the “About the technology” page, there’s this tidbit:

We wanted a database that could hold tens of millions of records, that would allow random users to modify its entries and keep a full history of their changes, and that would hold arbitrary semi-structured data as users added it. Each of these problems had been solved on its own, but nobody had yet built a technology that solved all three together.

So we created ThingDB (tdb), a new database framework that gives us this flexibility. ThingDB stores a collection of objects, called “things”. For example, on the Open Library site, each page, book, author, and user is a thing in the database. Each thing then has a series of arbitrary key-value pairs as properties. […] Each collection of key-value pairs is stored as a version, along with the time it was saved and the person who saved it. This allows us to store full semi-structured data, as well as travel back thru time to retrieve old versions of it.

This sounds really interesting. It also reminds me very much of Maya’s u-forms (pdf), aside from the fact that the identifiers aren’t UUIDs. Although I’m not really database-savvy enough to know much about the underlying infrastructure that makes any of this happen, so my interest is something like an ape staring at a power drill, but still, I thought it worth noting.

But Karen Schneider calls out what’s lacking. It is, after all, the style section, and there’s a lot of concentration on cocktails, clothes, and tattoos. There’s also a glossing-over of some stereotyping that deserves examining and lack of attention to the things that truly make librarians “hip”. Karen writes,

Jessamyn is of the hippest of the hip not because she routinely uses instant messaging, but because she is such a tireless advocate for small libraries and poor communities — the unserved, often voiceless communities many of us (including me) forget about when we get hopped up about some new new thing.

Right on. And she goes on to say,

I am cool in my subversive old-lady tech-loving the-user-is-not-broken way, and getting cooler all the time, and I count among my friends and colleagues librarians of all ages, dress codes, and evening habits.

Karen, if you want to identify as “old lady”, I’ll support you on whatever you want to be. But I have to say, also one of the coolest librarians I know of. Thanks for blogging.

When people tell me they’re too busy to blog, I ask them to count up their output of keystrokes. How many of those keystrokes flow into email messages? Most. How many people receive those email messages? Few. How many people could usefully benefit from those messages, now or later? More than a few, maybe a lot more. (Jon Udell, “Too busy to blog? Count your keystrokes.”)

The whole post is definitely worth a read.

I’ve been pretty absent from blogging lately, but I think I’ll take this to heart, both for this blog and for work-related emails that might be better served by an internally-facing blog. My company is reasonably compact (certainly in comparison to Microsoft, anyway!) but we still have a lot of challenges spreading information around when people aren’t all in the office together and communicating face-to-face. We’ve reached a size now where, even when we are all in the office, some people are still out of the loop on X, Y, or Z because they weren’t present at the conversation in so-and-so’s office, or over the lunchroom table, or wherever.

And speaking of not blogging in a while, it reminds me that I also have a post on Yahoo Pipes and Dapp and many related issues that I started ages ago — like when Yahoo Pipes debuted — that I still haven’t finished… Too busy…

I wrote the following in response:

On its face, this looks really awful, and I do agree that cars could be a lot more fuel efficient if cars companies wanted to make them that way and if people wanted to buy them that way.

But realistically, those high-efficiency cars from 1992? In general, they’re lawn mowers. They struggle going up hills. It’s no wonder no one wants to drive one. The fuel efficient cars from 2007? They’re real cars. The Yaris is totally suitable, and the Mini is downright sporty and fun.

Now, I’ll be the first one to agree with someone who says that gas is underpriced in this country, who says that we have an overwhelming bigger-is-better mentality that’s often not a good thing, that we are over-consumers and not concerned enough about conservation.

But, I don’t think that means everyone has to drive a small car. If you need a station wagon or an SUV or a pickup truck, that’s OK. And it certainly doesn’t mean that you have to drive a car that can’t make it up a hill.

What I really want is for the cost-to-benefit comparison to be more transparent, so that it becomes apparent to someone who’s driving a gigantic SUV what the tradeoffs between fuel efficiency and utility really are. Unfortunately, our current energy prices don’t take a lot of things into the accounting.

Flickr has just introduced “machine tags” (or “triple tags”). These build on existing geotags, which encode locations like this: geo:long=123.456. They’re three-part tags, with a namespace and a key-value pair, and you could use them to express all manner of things—like, for example dc:title=Othello. (There are also some semi-official uses of namespaces on tags in del.icio.us, like system:unfiled and filetype:mp3, and various users have used namespaces and triple tags on services like these without official support.) You might think of them as a kind of really lightweight RDF.

Triple tags really blow away the distinction between structured fields and freeform tags. This is important, because it’s a step along a road in which it’s easier for Joe and Jane User to make sense of complicated sets of data by sorting and filtering. Once you’ve become comfortable searching and sorting your tags, it’s not too much of a stretch to apply the same tools to more structured data. Sure, maybe it’s the same data that’s always been there, but now maybe Jane User could be better at manipulating it because she doesn’t have to understand “databases”, she just has grok “tags”, along with a little lightweight syntax. It’s just a different way of looking at the data, one that might prove more friendly. I know not all the tools are there yet, and I’m certainly not saying that everybody’s grandma is going to be putting machine tags on Flickr tomorrow, but I think this is a step in the right direction.