Some More Comments on CEO Pay

As an outsider (and a non-executive one at that) looking in, I can’t help but agree with Mark’s latest comments on CEO Pay. Obviously things are different in companies of 10,000+ employees and market caps north of $1 billion, perhaps that’s why I’ve been far more comfortable in start-up land.

Put another way, every hired CEO is looking to be in a position to look in the mirror , smile and tell themselves they have made it. They are living the American dream. The only way to do that is to grab as much equity equivalents as you can and do everything you can to get that stock price up as high as you can while periodically liquidating the stock and stuffing the cash in your bank account.

In my mind, it’s a game that CEOs can win in one of two ways.

1. Do a sufficiently good job at large company #1 so you can become CEO at larger company #2 (rinse and repeat)
2. Maximize individual equity (the so-called equity/lottery ticket zone) over the collective wealth of the organization (including those in the so-called cash zone)
3. Do both

It’s a simplistic model and I’m far from a CEO (just a pesky technology guy) so I can do nothing but speculate on the underlying motivations. That being said, the successful CEOs that we all hear and read about for the most part seem adept at #3, almost to the extent that their financial gains are often an afterthought. If it’s a matter of performance, equity should no doubt follow company successes.

Shareholders tend to ignore how much stock is given to management, they don’t ignore cash. Companies will always be a lot more stringent with their cash, whether its paid to the CEO or anyone else. CEO cash compensation will go way up, but total compensation will come way down. More importantly , CEOs getting paid huge sums in cash will stand out like a sore thumb when things arent going so well. They will be treated like everyone else in the cash zone and held far more accountable for their work.

Interesting thoughts. Accountability is king. Do a good job and you deserve to be recognized and rewarded appropriately, regardless of position.

It’s a Mac Mac World @ Open Web Vancouver

I just got home from Open Web Vancouver 2008 and I must say that MacBook Pro’s definitely reigned supreme amongst all presenters.

The crowd was slightly more diverse but still plenty of MBPs and even a handful of MacBook Airs.

Keynotes by Zak Greant and Tim Bray were very well received and very much inspirational in their own ways. Kudos if you can find me.

I’ll blog a bit more about the interesting specifics of the conference in the next day or two. One of the big takeaways for me surrounded microformats and the emergence of a more semantic web (or at least a more social web).

Storing Attributes as XML and Indexing with Lucene/Hibernate Search

We recently finished a short two week spike that involved using Hibernate Search (backed by Lucene) to index additional attributes that were persisted as an xml snippet stored in CLOB field.

Simple enough goals:

1. What does the performance curve look like?
2. Can we search the indexed attributes? (the biggie here was to still be able to do a SQL-like contains query)

The scenario:

Imagine the hypothetical Book table and suppose you wanted to provide the ability for end-users to define and store new attributes of the book without having to make structural modifications each time.

The typical solution to this would be to provide a more complex data model alongside the actual entity that allows a user to both define an additional attribute and provide values.

What we wanted to test was the situation where you want to persist 5,000 books at a time w/ upwards of 20 additional attributes. The hypothesis we evaluated was that encoding these attributes in XML and indexing with Lucene would be more performant and just as functional as persisting to a data model that stored attributes as additional records in some other table (or tables).

Our implementation:

@Entity
@Indexed
public class Book implements Serializable
{
    private Long bookID;
    private String name;
    private String xml;

    @Id @GeneratedValue(strategy=GenerationType.AUTO)
    @DocumentId
    public Long getBookID()
    {
        return bookID;
    }

    public void setBookID(Long bookID)
    {
        this.bookID = bookID;
    }

    /**
     * Getter for property 'xml'.
     *
     * @return Value for property 'xml'.
     */
    @Lob
    @Field(index=Index.TOKENIZED, store = Store.NO)
    @FieldBridge(impl=XMLFieldBridge.class)
    public String getXml()
    {
        return xml;
    }

    public void setXml(String xml)
    {
        this.xml = xml;
    }
}



public class XMLFieldBridge implements FieldBridge
{
    public void set(final String name, 
                    final Object value, 
                    final Document document, 
                    final Field.Store store, 
                    final Field.Index index, 
                    final Float boost)
    {
        if (value == null || "".equals(value))
        {
            return;
        }

       for (Attribute attribute : xmlParser.parse((String)value)))
       {
           Field field = new Field(attribute.name, attribute.value, store, index);
           if (boost != null
           {
              field.setBoost(boost);
           }
           document.add(field);
        }
    }
}

There really isn’t much more than this. Some of the details (ie. XML parsing) have been left out for brevity. Hibernate Search requires you to use annotations for the Lucene aspects but you’re free to using traditional xml-based (HBMs) configuration for ORM mappings.

**
**

The results:

By and large the results were pretty good. We did setup asynchronous indexing (but on the same physical machine) which worked quite well. There was still fairly significant overhead associated with the actual XML parsing that was somewhat unavoidable. Once the XML was parsed the indexing did occur post-transaction which didn’t add any additional overhead.

Performance was pretty good too (roughly constant time depending on the amount of data being indexed). Even with indexing on the same box (5 indexing threads) the persistence time for 5,000 books and 20 extra attributes was in the same ballpark as persisting multiple records in the more complex data model.

If indexing were offloaded to another box, I’m confident that it would have been significantly faster (this was somewhat verified by inserting a Thread.sleep() between test iterations to sure that the backgrounded indexing had completed prior to starting another test iteration).

The actual post-transaction indexing wasn’t particularly speedy (tokenizing and indexing). There was significant time spent indexing when re-indexing 500,000 -> 1,000,000 records.

What actually proved to be a show stopper was the inability to do a SQL-like contains query. Lucene does has an analyzer that allows you to do a starts with query but nothing more complex than that. To Lucene’s credit, it isn’t particularly efficient to do this in most databases. If worse came to worse, I suspect we could have implemented our own Lucene analyzer.

Hibernate Search was quite easy to work with. Using annotations made sense and once we got past a few maven annoyances life was pretty smooth. Documentation was quite well done and covered most everything we wanted to test.

It doesn’t support the latest release of Lucen but I trust that will be coming at some point. Unless the API has changed, you may be able to use a newer version of Lucene but without access to it’s APIs via the Hibernate SearchFactory?