Jay Pipes

In the Columbus, Ohio, airport right now waiting to board my plane to Salt Lake City. Tonight, I will be speaking on Join-Fu at the PLUG (Provo Linux User Group). Ryan Simpkins asked that I speak "at a very advanced level" and so that's what I will be doing! We'll see how the PLUG members handle my new set of slides on Intermediate Join-Fu that I have been working on...

I will post the slides a little later today after the PLUG meeting. I'm really looking forward to meeting Ryan and the other folks from PLUG and getting a short tour of Omniture, a large web analytics company that uses MySQL (no, it's not Overture. Totally different. One of those is now extinct.)

Peter has suggested this before, and I think it's a great idea. Basically, Peter's gripe is that currently, there is no way for community members to propose new worklog tasks for either the MySQL engineering team, or more likely, the external contributor community, to work on.

As a little background, the MySQL Forge Worklog system is simply a public-facing, read-only interface which shows our internal development roadmap tasks. We have an internal Worklog system which our engineering teams use to track progress along various milestones and projects. The Forge worklog shows all tasks which have been marked as "public" in the internal system. You may be surprised to learn that more than 90% of our internal worklog tasks are marked public.

In the public Worklog system, you'll see tasks that have been assigned to a developer, tasks that are unassigned or in progress, and tasks that have been (tentatively) allocated to appear in future versions of MySQL (6.x, 7.x, 8.x, etc). It is interesting to browse the tasks on the worklog to get a picture of what the future of the MySQL server holds.

But, while it is great to be able to comment on these worklog tasks (simply go to a task, login to the Forge, and post a comment in the area to the right), that only goes so far. It would make things interesting if the community could truly drive the development roadmap itself by proposing and advocating for their own worklog items. And, I plan on making this a reality.

Taking Action - A Community-Driven Worklog

My wonderful team lead, Giuseppe, has given me the ability to focus on coding the Forge and PlanetMySQL this quarter. This means that I have the time to work on a community-driven Worklog system. Below, I give my thoughts on what I will add to this system, along with the help of Robert van der Mast, who I am mentoring in a Google Summer of Code project related to the Forge.

Keeping a Single Interface

One thing I want to ensure is that as much as possible, we keep to a single interface for viewing and editing all worklog information. My vision to enable a community-driven worklog system is to have a interface which adds new Worklog tasks into the public MySQL Forge worklog database and marks such additions as "PROPOSED BY COMMUNITY" or something like that, but having such tasks "mixed in" with the rest of the Worklog Tasks on the Forge. I don't really see a need for a separate "community worklog" module or similar. Agree? Disagree?

Ability to Assign the Task to an External Contributor

So, along with being able to propose a new worklog task, we would also need to be able to assign the task to an external contributor, so that people viewing the task in the worklog system would be able to know that the task is under development by someone. How should such a system work? Should we limit the ability of assignment to only Forge users? This seems to be a more reasonable capability than simply allowing a text box for entering someone's name... Should there be an ability to add more than one contributor? Should the list of assignable contributors include MySQL developers as well? What are your thoughts?

Your Feedback

I am very interested in getting feedback about this. Feel free to add a comment to this entry or email me at REVERSE('moc.lqsym@yaj'). Thanks, Baron, for the email obfuscation...

I've been thinking about this for a while now, sparked by Ronald Bradford's brainstorming about a global hackathon, and I think it's something that is completely doable, and would have great benefit for MySQL and also the external contributor community. The idea is similar to the PEAR Bug Triage event that occurred a few weeks ago. Basically, the idea is to set aside a time period (1 day, 2 days?) in which the global external contributor community and the MySQL developers come together in a virtual setting and tackle a list of fixable, but low priority, outstanding bugs in the MySQL server and client.

Current Roadblocks

So, what needs to happen for this to become a reality? These are the things I see as current barriers to the event, and issues I hope can be addressed in short order so that a MySQL Bug Triage event can happen sometime in late Western hemisphere (see, Stewart and Lachlan, I don't forget you guys!) summer or early fall.

• BitKeeper

  Yes, the age-old issue with our current source-control system. While it is true that contributors can use the free bk client, one of the issues folks have is the lack of the free client's ability to produce a patch/changeset that can be merged into a real tree. So, BitKeeper is currently a barrier to external contributors that would participate in a triage event.

• The current MySQL Contributor License Agreement (CLA)

  As Mark Callaghan notes in the oursql-sources Google Group,

  I agree with Baron, the CLA is a barrier. Why don't they use the Apache CLA? Who is able to determine the differences between the MySQL and Apache CLAs? And who is willing to pay for a lawyer to do so? One CLA has an active developer community, one does not.

  My suggestion is to have two agreements, one which would make it possible for Sun/MySQL to be granted or purchase rights for the contribution to be included in the commercial distributions of MySQL, and another CLA option which would make it possible to grant the contribution only under a shared-copyright, OSS-license-only agreement.

  Paul McCullagh takes my suggestion one step further by saying:

  So my idea is to collect the rights to contributions under a type of "MySQL foundation". Community contributors would then sign a CLA which is issued by the foundation.
  If MySQL/Sun wants pages from the foundation tree, then it buys the commercial rights to those patches. The foundation then distributes the income amongst the contributors.

  I think either option is a viable one, but resolving the current legal/licensing roadblock with the existing CLA is essential to getting a MySQL bug triage event going. Either we fix it, or important and knowledgeable contributors like Mark, Peter, Baron, Vadim and Paul cannot contribute.

OK, so assuming we resolve the above issues, here is what I think needs to happen in order for the MySQL Bug Triage event to be successful.

Determining a List of Fixable Bugs

Georg Richter and his team have already done a decent job of identifying bugs and worklog items that are deemed easy to fix and not sufficient-enough priority for the MySQL engineering department to be working on. If you look at this list of bugs and worklog items, there is a significant chunk that I think the community can knock out in a couple days, given adequate preparation and working knowledge from the MySQL development organization. So, what preparation and working knowledge is needed?

Preparation

Although there is a bunch of material on the MySQL Forge wiki on contributing to MySQL, the information is a) not very widely known, and b) not particularly geared towards C/C++ developers looking to fix bugs and/or contribute a feature.

Before a MySQL Bug Triage event, the external contributor who have experience working with the MySQL source code need to do the following, so that other interested community members can quickly get a development environment up and running:

Write detailed instructions for setting up a development environment

If an interested community member is to have a fighting chance at being able to fix bugs in the MySQL server, then she must be able to accurately and easily set up a working debug/development environment on her local machine. Therefore, we need experience MySQL developers and contributors to use the Forge wiki to create detailed instructions for establishing development environments on MacOSX, Windows, and Linux platforms. Who can volunteer to work on these pages? Following the links below to each platform will take you to the stub pages I set up on the Forge wiki for folks to assist in filling in.

There are a number of decent instructions on the Forge wiki already which cover topics regarding building MySQL on various platforms. Perhaps the best is Iggy Galarza's wiki page on Building MySQL on Windows What is needed for the Bug Triage event to be successful is all such information about building and debugging MySQL on various platforms needs to be consolidated, structured, reviewed, tested and organized, so that a contributor can quickly get up to speed. Who will answer the call?

• Help with Contributor Development Environment: Linux
• Help with Contributor Development Environment: Windows
• Help with Contributor Development Environment: MacOSX

Organize and Improve the Instructions for Producing Test Cases

The forge wiki currently has a section on How to Write Test Cases, but unfortunately, this wiki page is of a MySQL University session and is not widely known and doesn't contain very detailed information that steps a contributor through everything they need to know in order to create a test case that covers the bug they are working on. A while a ago, I wrote a blog entry about this very topic, but I need help from contributors and the MySQL development team to verify that all the information in that article is currently accurate and porting the article into the Forge wiki. Who can assist me in this regard?

Can a Bug Triage Event Be Successful?

The above are just my thoughts on making a bug triage event successful. What are your thoughts about this?

Last night, I was fortunate enough to give a presentation called "Join-fu: The Art of SQL Tuning" at the local Columbus PHP Meetup group. There were about 14 people there, and it was a great time. Lots of good questions! Anyway, I created a new slide deck with some new material on LEFT-join-fu and managing many-to-many relationships that folks might find handy. Below are the slides in OpenOffice Impress and PDF format. Enjoy.

• Join-fu: The Art of SQL Tuning (OpenOffice Impress)
• Join-fu: The Art of SQL Tuning (PDF)

that the "lightning bloggers" at the MySQL conference achieve total dominance of the "Top 30 Most Active" list on PlanetMySQL. I have so many things going on at the conference that I don't often find time to blog and so my name drops off the active list as fast as you can say "online backup"...

OK, in all seriousness, this post is actually about something... Our friend Julian Cash will be at the conference tomorrow, taking photographs, in his trademark fashion, of conference attendees. His work can be found all over, including Brian Aker's site, and he's been at various open source conferences around the world. He'll be in the Bayshore Room, on the Mezzanine level of the hotel, between 10am and 2pm tomorrow, so come and introduce yourself, play with all the toys he is sure to have around, and get photographed!

In case anyone is wondering, I will be at two birds of a feather sessions today, one with the PHP/Zend crowd and another with the good folks over at Wavemaker.

MySQL and PHP: Joined at the Hip

It's no secret I'm a huge fan of PHP, so it should be no surprise I'll be joining the fun over in Ballroom H, from 7:30-8:30pm tonight to discuss all things PHP and MySQL.

Be a Web 2.0 Rock Star

OK, I'm a sucker for competitions, and this one definitely looks like fun!

It's Tuesday evening, tonight, April 15, at 8:30 in Ballroom E. For rock stars, compete against other Ajax experts to build the coolest Web 2.0 app in 30 minutes and win your very own copy of the Rock Band video game. For newbies, get an up-close look at the latest Ajax and Web 2.0 technologies with Anand Pandey of MySQL and Geremy Cohen of WaveMaker.

So, I have been mistaken in thinking that the MySQL Conference web system was set up for providing online feedback for speakers. Apparently, there is a 3rd party online service in use for providing speaker feedback. The link to it is available here.

Cheers. I'm off to run more benchmarks for tomorrow's presentation with Tobias.

Of all the practical jokes on April Fool's Day, only one actually got me. Yes, Ronald Bradford's post yesterday describing a visa situation and having to return to Australia actually got me looking up immigration attorney numbers in the US that could help as well as emailing him my heartfelt sympathy about his situation. I even started planning for who might replace him at the MySQL Conference, as it sounded like he was not going to be there.

In response to my email, he wrote back:

April Fools Dude!

to which I wrote back:

Oh, you totally suck.

So, Ronald, you have the dubious honour of duping me (among others, I see). Congrats!

At 1:30pm EST today, I am giving my much-anticipated talk on Using Open Source Projects for Personal Profit at the first annual HabariCon, which is a celebration of uniting the user and developer communities in friendship and collaboration. I am expecting 80-120 people to show up for my talk, which focuses on how you can contribute a bit of code early on in an open source project's life and claim to be a valuable member of the committer society, thus increasing your overall karma in the open source community and landing you a higher-paying job. It's must-see learning for anyone interested in the economics of Cabal-coding. See you there!

Feel free to join the Habari Cabal and others on Freenode #habari and learn invaluable tips from some of the best open source community leaders in the world!

For those of you who have been stuck under the proverbial rock for the last couple months, the MySQL Conference and Expo is just around the corner. The conference's tutorial day is April 14th, which means if you haven't registered yet, you should definitely do so. For a 20% discount, email me at my first name at mysql dot com. Register soon because tutorials are already selling out or entirely sold out.

Hitchhacker's Guide to the MySQL Conference

PHP Developer's Edition

This blog post marks the first in a series of guides to the conference that we'll be publishing in the next couple weeks leading up to the conference. The guides are intended to help a specific audience find the sessions that will likely be the most interesting for them. The first edition of the Hitchhacker's Guide is for PHP and MySQL developers. As PHP is dear to my heart, I'll be writing this one myself. Others will be following up with other editions, and the guides will all be posted on the MySQL Forge's wiki pages for the conference.

The Keynotes

If you attended last year's conference, you will undoubtedly remember some of the most entertaining and memorable keynotes ever given at a MySQL Conference (or any other conference, IMHO...). Guy Kawasaki dazzled, Eben Moglen awed, and a panel of DB Egos amused the keynote audiences. This year's lineup is, in a word, amazing, and any PHP developer worth their salt is going to want to hear all of them. Here, however, are the three that PHP-ites may find the most fascinating...

A Head in the Cloud - The Power of Infrastructure as a Service is a keynote delivered by the reknowned Werner Vogels, CTO of Amazon.com. The keynote is about the challenges faced while building Amazon.com and the future challenges this industry is bound to face. So, why should PHP developers want to hear about Amazon.com if that site isn't a developed in PHP? Because the talk is predominantly about the challenges that scalable environments face. Regardless of whether Amazon.com is written in PHP, developers using PHP still face similar challenges in scaling to meet the demands of modern applications. So, listen and learn from one of the best.

Copyright Regime vs. Civil Liberties. Rick Falkvinge is the head of the Swedish Pirate Party... What on Earth does this have to do with PHP developers? Everything. As PHP developers, we use a language that is almost 100% community-driven, and not backed by a single corporation. It is a defining characteristic of the PHP language itself, and the community as a whole, that no one person owns PHP. It's a communal thing! Well, Rick will be speaking about the problems with copyright laws as they exist today, and what things are being done to address those problems. In light of the recent debates regarding PDOv2, this is something all PHP developers should keep an eye on!

A Match Made in Heaven? The Social Graph and the Database. Jeff Rothschild, VP of Technology at Facebook. Facebook is a huge PHP shop. Facebook is a huge MySQL shop. Facebook is huge. Facebook. Enough said.

Tutorial Day (Monday)

My recommendations would be to catch Giuseppe Maxia and Jan Kneschke's full-day tutorial on the new MySQL Proxy, Bill Karwin's SQL Anti-Patterns tutorial, or Brian Aker and Alan Kasindorf's Memcached Complete tutorial. Any of them is a great bet. I'd recommend Ask Hansen's Real-World Web tutorial, but it's been sold-out for a while...

Sessions to Catch

Tuesday, April 15th

After the keynotes on Tuesday morning, you've got a whole lot of choices — some of which will be exceedingly difficult to make. In fact, some of these sessions are going to be standing-room-only, so if you really dig(g) a session, then show up early for it and get a seat. In the first session block, there are two sessions of particular note to PHP devs. First off is Sebastian Bergmann's Testing PHP/MySQL Applications with PHPUnit/DbUnit. I personally have marked this on my personal conference calendar, as I am very much interested in learning how to harness DBUnit for the database-centric tests for MySQL Forge. In addition, during the same time slot is Mike Zinner from MySQL talking about the new MySQL Workbench GUI toolkit in MySQL Workbench: The Ultimate Database Design Tool for Developers. I know my friend Cal Evans will want to be in that one...

Tuesday's 11:55am and 12:40pm time slot is perhaps the most difficult choice for the PHP developers. There are five sessions that will interest developers using PHP. Perhaps the most popular will be Robin Schumacher's talk about the future of MySQL and Domas Mituzas' talk on Practical MySQL for Web Applications. However, popularity aside, Beat Vontobel's "The Lost Art of the Self Join" should be amazing since he will be solving a Sudoku puzzle entirely in SQL.

Another difficult choice will present itself in the after-lunch session block. Seasoned MySQL conference speaker Mike Ho will be presenting his ideas about object-relational mapping for the PHP developer in "Beyond SQL and ORM: Object-oriented MySQL Querying in PHP". At the same time, you may want to check out Sigurd Magnusson, of Sliverstripe fame, talk about Upgrading to Elegant and Versatile Database Architecture Using PHP5 Data Objects

Rounding out Tuesday's must-see sessions are Baron "the Schwartz is with you" Schwartz' EXPLAIN Demystified, a talk which every developer interested in the performance of their SQL queries should attend, and Replication Tips and Tricks with MySQL engineer Lars Thalmann. Anyone using replication and PHP should attend this talk. Finally, Joe Stump from Digg.com will be talking about implementing Service-Oriented Architecture with PHP

Wednesday, April 16th

Wednesday also holds a number of choices for PHP devs to make. In the morning, I'd recommend catching either Roland Bouman's sure-to-be-cool "Grand Tour of the INFORMATION_SCHEMA" or Giuseppe's talk about MySQL Sandbox. If you use multiple database servers, for testing, staging, development, etc., I would highly recommend checking out the Sandbox to alleviate the pain of multiple MySQL instances. Well worth it.

Next I suggest heading over to catch Colin Charles talking about what the MySQL Community is all about; what challenges it faces, how it functions and grows, and how you can contribute to it. After lunch, you've got a tough decision. You can either come see me and Tobias talk about Microbenchmarks for MySQL (sure to be fun!) or view Domas Mituzas' talk about handling character sets in MySQL and PHP, or if you are interested in an upcoming storage engine from MySQL, you can see Monty talk about the new Maria engine...

In the afternoon, be sure to check out Baron's talk on the MySQL Query Cache or see two of our web team talk about Using Frameworks in Web Application Development. In addition, find some time to attend Tom Hanlon's 90 minute talk on Benchmarking and Monitoring MySQL. Word of warning: Tom's talk will be standing room only. Get there early.

At night, don't forget to attend the MySQL Quiz Show, MC'd by yours truly, and a cocktail party sponsored by Sun directly afterwards. Get your Quiz on.

Thursday, April 17th

The final conference day is chock-full of meaty stuff for the PHP developer. After the morning keynotes, seek out George Trujillo and Alexander Nozdrin's talk on Stored Routines or go hear Jeremy Cole and Eric Bergen talk about hardware considerations for scaling MySQL. Either talk should be dynamite.

Follow that up with the always-entertaining Damien Seguy's MySQL Hidden Treasures or Mark Callaghan's Helping InnoDB Scale on Servers with Many CPU Cores and Disks. Finally, make sure to get into the ballroom to hear Dathan Pattishall talk about his techniques for achieving massive Real-time Concurrent Writes at flickr.

Register Now for the 2008 MySQL Conference and Expo!

As you can see, there's a ton of options for the PHP developer at this year's conference. If you haven't registered yet, do it now! Email me for a 20% discount code!

Just a quick note to let folks know that today I coded up functionality to allow MySQL Forge users to re-vote on something they have already voted on. To re-vote, simply navigate to an item you've previously voted on, and click on the number of stars you wish to change your vote to. All done.

Also, I have corrected the ranking of items in display lists to take into account the number of votes an item has received (you're welcome, Giuseppe.). So, if both Project A and Project B have an average rating of 4 stars, but Project B has ten more votes than Project A, Project B will rank higher than Project A in the display lists.

So, get on to the Forge and start voting for your favourite projects, code snippets, and worklog tasks! Somebody other than Giuseppe should vote for Giuseppe's contributed tools!

As Giuseppe and Kaj have blogged about, the student proposal period for the Google Summer of Code is underway from today through March 31st.

The new MySQL Forge has a wiki page with suggested MySQL-related GSoC projects that a number of MySQL engineers and community team members have proposed. The projects include C/C++ projects in the MySQL server as well as other projects for the non C gurus among you, including some projects in Lua for the MySQL Proxy.

PHP Hacker? Know MySQL? Hack on the MySQL Forge for Google Summer of Code

If you love PHP and MySQL, there are other ways you can participate in the summer of code as a MySQL project submitter. I will be open-sourcing the new MySQl Forge code base shortly (only technical issues remain...). I would love to see some interest from the PHP and MySQL community in enhancing the new MySQL Forge with new functionality. Therefore, I have created a section on the Forge wiki with a list of proposed projects for GSoC students. Here is the list so far, but feel free to propose anything else you'd like to see implemented on the Forge:

• Develop a "karma system" for Forge users that is integrated with the MySQL forums, mailing list activity, Forge wiki activity, and code snippet creation
• Enhance the user interface so that Forge users can "watch" a project, code snippet, worklog task, etc. Preferably a nice AJAX widget similar to what the rating button currently is
• Develop an RSS and Atom feed system for the Forge "new items" (tasks, projects, code snippets, people?)
• Integrate PlanetMySQL with the MySQL Forge People section
  • The user "blogs on PlanetMySQL", for instance...
• Integrate OhLoh.net statistics into the Projects area
• Expand test-suite for MySQL Forge and better integration with PHPUnit3

Also feel free to email me at my first name at mysql dot com if you wish to chat or brainstorm ideas on PHP and MySQL GSoC projects.

For those unaware of what the Google Summer of Code is, here is an FAQ

Just a quick note to let folks in the Bay Area know about a cocktail party celebrating the Sun acquisition of MySQL. Come meet both MySQL and Sun folks, ask them questions, and most importantly, just have fun. The reception is at Jillian's @ The Metreon, 101 Fourth Street, San Francisco, and starts at 5:00pm. You can download a flyer that has directions to the venue.

So, this past week I was in Montreal attending the annual PHP-Quebec conference. It was my first time up in Montreal and I have to say, it's a fun place! It was great meeting so many interesting folks and my session was jam-packed and a lot of fun.

One of the most interesting conversations at the conference was regarding PDOv2 and what, if anything, is going on with it. Lukas Smith raised the spectre of PDOv2 (after a tiny little prod from me. ) at the Database Panel discussion with me, Kitman Cheung from IBM, Kuassi Mensah from Oracle, and Bob Bernier from PostgreSQL. It seems that Microsoft and Oracle have responded to the PDO mailing list and we are awaiting some word from IBM. Kitman promised to look into it and have someone from IBM respond shortly. In general, besides the active community members in attendance, not too many folks were that gung-ho about PDOv2, and it seemed to me that the suggestion that the community should simply enhance PDOv1 with better docs, test coverage and standardization of interfaces was the best option...anyway, more on that later.

Here are the links to the slides which are identical to the slides I gave at CodeMash this past January, but I will post them anyway. Cheers, all!

• OpenOffice.org Impress presentation
• PDF presentation

I often get accused, especially by my European friends, of not being a true geek for two reasons: I don't carry an expensive camera with me at all times — nice Fisheye lens, Kaj! — and because I don't indulge in video games. Well, for all you Jay-ain't-a-true-geekers out there, I offer two photos today to prove my geekhood.

As some of you know, I live in Columbus, Ohio, with 2 dogs, 2 cats, and my lovely wife Julie. Last night, we got hammered with the most snow we've seen in March in over 40 years. By my estimation, there is about 18 inches of snow on the ground. The photo to the right was taken this morning, while the snow was still coming down, from my back door. You can see my garage and the ground covered in snow, but the most telling part of the photo is the amount of snow on the table on our back deck. I think there's at least 18 inches of snow on it. At the very bottom of the photo, you see the height of the snow on the ground; that strip on the very bottom is my back door threshold...

So, what does this mean for me? A snow day? Heck no! I'm a MySQL employee and I work from home. I don't take snow days, of course, so while my wife has closed her yoga studio, I'm snug and warm and working from home on a Saturday on my slides for next week's presentation at PHP-Quebec on "Performance-minded MySQL for PHP Developers".

If you're in the Montreal area and want to go, I have two free passes to give away since Sun/MySQL is a sponsor for the event; we're sponsoring a cocktail party and job fair on Wednesday evening. So, if you're interested email me at my name at mysql dot com.

Which kind of leads me to my next photo... This one is of my dining room table covered with paper. What are all these pieces of paper? Well, each and every paper on the table is a separate form or agreement I needed to sign and fill out as part of becoming a Sun employee. Hmmm. I don't think I remember having to sign so much paperwork when I joined MySQL over two years ago. Oh, how I yearn for the simple days of a small company!

Hope that the photo to the left doesn't scare anyone away from working at Sun! You know, we're hiring...