Sep 19 2014

A Manager or a Community Person: #PASSVotes

I am running for the PASS Board. You can read more about what I’ve posted and what others have to say here on this page.

Today I want to ask you a question. Should someone on the board be a manger or should they be a community person? Let’s avoid the easy answer of both for a moment, not because that’s the wrong answer, but because it’s the right one. Clearly you need a mix of these skills to be on the board. But, where you fall on my simplistic question could determine the kind of person you want to vote for on the board. Let’s discuss it a bit.

According to Wikipedia (deal with it), a board of directors is “meant to oversee the activities of a company or organization.” Well, my question is answered then, a manager is what’s needed. We’re done. Boy that was a stupid question Grant. But hang on a second. Let’s take a look at the mission statement of the PASS organization:

Empower data professionals who leverage Microsoft technologies to connect, share, and learn through networking, knowledge sharing, and peer-based learning.

Now what do you think? Do you just want some manager type, or maybe, just maybe, having a data professional directly involved in peer-based learning, for example, answering questions on SQL Server Central. A person heavily involved with knowledge sharing, maybe, running a blog and presenting webinars.  As for networking, let’s talk at one of the live events where I’ll be presenting technical talks. In short, I think I’m well positioned to answer the PASS mission statement. I have team lead and project lead experience. I’ve also helped run volunteer organizations. But, what I really am is a community guy. My passion and my interests line up directly with the PASS mission statement. This is a large part of why I’m running for the board. I believe in what the organization does and I want to help it do more.

So, I’m pretty sure we need managers, but I really do think we need community people too. If you agree, please consider voting for me, Grant Fritchey, during the PASS Election between the 24th and the 29th of September. Keep on an eye on your inbox for your ballot and your chance to vote. For more information about the PASS election and information on the other candidates, go here.

Sep 17 2014

I Am Running For the PASS Board of Directors: #passvotes

This year, I submitted my application to run for the PASS Board and it was accepted. This then is my announcement to all of you and the beginning of my campaign for election. My name is Grant Fritchey and I’m running for the PASS Board.

It’s traditional to either make all sorts of promises for the things you’re going to do or to attack your opponents. I’m not going to do either. I don’t have any grand promises to make. I’m not going to attack my opponents because I know and respect them. I’m voting for James Rowland-Jones myself and so should you. Let’s do this instead. Let’s talk about PASS, you, me, your career, mine, and how PASS can change your life.

PASS changed my life for the better. Like many people, I’m constantly trying to learn how to do my job better. In 2005, I got the opportunity to go to the PASS Summit in Dallas. Just like many of you, I attended sessions and learned all sorts of things. It was great. Right there, PASS can change your life by teaching you more about SQL Server and related technologies. Not life changing enough? Not for me either. While there, I met some volunteers who were helping to run the organization, so I decided to volunteer too. One thing led to another and suddenly, I knew people from all over the country and our “family reunion” was at the PASS Summit. Yeah, a network. No, not to look for a new job. That’s not the primary reason for having a network. The primary reason for having a network is an extended set of knowledge. I know some things my friends don’t. They know some things I don’t. We can help each other. PASS supplies the medium through Summit, SQL Saturdays, the BA Conference, webinars and the local user groups it supports to provide you with the means to build your own network. That’s pretty life changing, right? Yeah, I’m somewhat underwhelmed too. I also started speaking at the local user group that we set up and I started submitting to speak at PASS and finally made the big stage. Oh boy, I get attention. Yeah, that’s one reason for doing it, but, at work I started presenting a lot more too. I was presenting stuff to my team, my boss, their boss, and ultimately to large sections of the organization. Yeah, PASS can help you there too, again through SQL Saturdays, your user group, and maybe, for a few, Summit. There are also opportunities for you to write, record videos, help run the program committee, all sorts of volunteer positions within the organization that will assist you in building your skill set and your presence which will all be completely applicable inside and outside work. I’ve done many of these and they lead me to becoming an MVP and getting a fantastic job for an amazing company. I absolutely attribute all that to PASS. PASS changed my life, and it can change yours.

So, back to this election business. Why am I running for the board? Because I want to do two things. First, I want to help keep the organization going strong and I think it’s time that I volunteered at a higher level to make this happen. Second, I want you to find and take advantage of the opportunities that PASS offers so that it can change your life too. We need to get the word out to a lot more people in order to show them what the organization can do for them. It’s about teaching technology, sure, but it’s also about networking and mentoring and personal growth. This organization can, and will, change your life. The structures are there. You just have to make the choice to reach out and grab those opportunities. I want to share this with others. Those are my reasons for running.

The only promise I’ll make is that if I get on the board, then the passion I bring to, well, everything, will be applied to whatever work I can do there to keep this organization running and get you the opportunities to change your life. Please vote for me. My name is Grant Fritchey and I’m running for the PASS Board.

For more information about the election, to see if you are eligible to vote, to see the other candidates and all the rest of the details, go here.

Sep 16 2014

PASS Summit 2014 Pre-Conference Seminar

I’m putting on a pre-conference seminar (also known as a pre-con) at the PASS Summit this year. I’m really honored to be able to present this and I’m pretty excited about it. So, if you want to talk query tuning, let’s get together at the Summit. For a few fun facts about the event, check out this Q&A over at PASS. To register for the event and my pre-con, go here now.

Jul 21 2014

Victims of Success

I took part in the PASS Summit 2014 selection committee this year because I was really curious about seeing how the sausage gets made. I’ve seen how actual sausage gets made and I still eat sausage.  Despite a few hiccups and communication issues, internal and external, I think the selection process for the Summit went really well this year. But, there was still some controversy. Being a naturally pushy person, I got involved in the controversy, for good or ill, and subsequently have had conversations with many people about the selection process (which, reiterating, I think went extremely well overall). But, the one thing that kept coming up over and over was a simple question:

How come I/PersonX didn’t get picked?

The easy answer is because you/PersonX had a horrible abstract. But you know what, in probably most cases, that’s not true. Good abstracts by good people didn’t get selected, so what the heck? I think the more complex answer does not go back to the selection committee or the selection criteria or the selection process. Do I think some improvements are possible there? Yes, and I’m putting my foot where my mouth is (or something) and joining the committees to try to make some tweaks to the system to make it better (and really, we need tweaks, I want to repeat, risking ad naseum, the process went well and worked great and I’m happy I took part and I think the outcome is pretty darned good). No, the real problem lies elsewhere, SQL Saturdays.

I’m not saying SQL Saturdays are themselves a problem. What I’m saying is that PASS took on the whole SQL Saturday concept for several reasons, one of which was for it to act as a farm team for speakers. This will be my 10th Summit. Looking back to 10 years ago, while I really loved the event, oh good god have the speakers improved. I remember sitting in sessions with people who were mumbling through their presentations so much that, even with a microphone, you couldn’t hear half of what they said. Slide decks that consisted of 8-12 pages of text (yes, worse than Paul Randal’s slides, kidding, don’t hit me Paul). Speakers who really, clearly, didn’t have a clue what they were talking about. It was kind of rocky back then. I learned my second year that you had to talk to people to find out, not just which sessions sounded good, but which speakers were going to present those sessions well enough that it would be worthwhile. Why were there so many weak presenters? Well, because there was almost nothing between speaking at local user groups and speaking at Summit (I made the leap that way). There were a few code camps around, a couple of other major events, various schools and technical courses, and Summit. I don’t know how the old abstract/speaker review process worked (and I apologize to whoever read my first abstract because I know now just how horrific it was and I’m so sorry I wasted your time), but I’m pretty sure they were desperate to get enough submissions that sounded coherent with a speaker attached that probably could get the job done. Not any more.

Now, people are getting lots of opportunities to present at SQL Saturday events all over the world. And it’s working. We’re growing speakers. We’re growing good speakers. Don’t believe me? Then you go to two or three events in a month, sit through 8-12 sessions, mostly by newer people, not Brent Ozar, not Denny Cherry, not Kim Tripp, and you review them, each, individually, then go back and try to pick the best one. Oh yeah, there’s going to be a few dogs in the bunch, but overall, you’re going to find a great bunch of presentations by a great bunch of speakers. Our farm system is working and working well. But there’s a catch.

Because we have upped the bar pretty radically on all the introductory level speakers (and if you’re thinking about presenting, don’t let that slow you down, everyone starts at zero and goes up), that means the competition at the top (and yes, I do consider the Summit the top in many ways, not all, see SQLBits) is becoming and more and more fierce. That means, my abstracts probably need quite a bit more polish than they’re getting (and so do yours) because there are a whole slew of speakers coming up that are writing killer abstracts. That means I need to really be concerned about the evaluations (despite the fact that I get dinged because the stage is low, the room is hot/cold, lunch didn’t have good vegetarian choices, England left the Cup early, all outside my control) because there are new speakers that are knocking it out of the park. In short, you/I/PersonX didn’t get picked because the competition has heated up in a major way.

In short, a sub-section of the community, defined by those who wish to speak, are victims of the success of the farm team system as represented by SQL Saturday. On the one hand, that sucks because I now need to work harder than ever on my abstracts, on the other, we’re going to see very few instances of really bad presentations at Summit. We’ve improved the brand and the community. It’s a good thing.

Jun 26 2014

Passion

I know I tend to be overly passionate. It’s something that has gotten me into trouble in the past. It’s also probably a huge factor in the things I’ve been able to accomplish in life. I’m bringing it up at this time because I think passion is causing some conflict within the community around the Professional Association for SQL Server (PASS).

On the 25th of June just past the announcements went out for the sessions accepted at the PASS Summit 2014. I found this stressful and exciting two ways. First, and for me personally, most importantly, because I had submitted several sessions and I desperately wanted to speak at the PASS Summit (I’ve spoken there every year since 2008 and I’ve made the Top 10 sessions two years in a row, for which I’m truly grateful, back to our story). Second, because this year I wanted to help make a difference so I volunteered on the selection committee (and I was on a committee other than one I submitted for, I didn’t influence selection there at all). I wanted to get my sessions accepted, and I wanted to see the work I put in on display. Happily, both occurred. But, the day was marred.

Let’s sidetrack (again) for a moment. I consider myself to be just a guy, a DBA, a developer, an IT pro. It’s what I’ve been doing for 20+ years (yeah, I’m old) and I’ve been relatively successful at it. But, I’m also a Microsoft MVP, a published author, frequent blogger, and an international speaker. I attribute most of that stuff, not to any great ability I have, but to a lot of luck, a lot of hard work, and, here’s the kicker, to my involvement with PASS. Go back ten years, I went to my first Summit down in Dallas, TX. I attended sessions and went back to my hotel room, except one night. During that day I had spent a little time chatting with a company and they invited me to a party they were throwing that night. I went. And I met some people. They were just DBAs and developers, just like me, but, they were also involved in the organization that put on the event, PASS. I liked these people. So, I started volunteering which led to another Summit and another and writing and speaking and… well, let’s just say, getting involved was a good thing. Being passionate about it all paid off, literally and figuratively. I really do owe PASS and the people that make it up a lot.

So, there are a lot of passionate people in this little gang of ours. And some of those passionate people didn’t like the outcome of the selection process. Being passionate, they voiced their opinions. LOUDLY. At length. Some of what they said had merit. Some of what they said was just hurt feelings. Some of what they said was a complete misunderstanding of how things worked within the committees and the selection process. But a lot of passionate people, who care about PASS, argued for a little while about the Summit selections. And, being a passionate guy, I took part. A lot of the work I did for the committee wasn’t making the light of day (more on that later, maybe, depending on how some internal communications turn out) and I was quite passionate about that. I don’t know this for a fact, but I suspect pretty strongly that my passion, what’s more, my public passion, around this topic made some people angry. I’m positive that others passion for the topic, regardless of their causes and the rightness or wrongness of their cause, definitely made people angry. Here’s where I get in trouble.

Get over it.

If we didn’t care about PASS and what the organization has done for us, and how we’d like to help it, and help others, and grow it, and reward ourselves (because I do believe everyone is fundamentally greedy, might as well acknowledge it), and just plain replicate the experience for others that I’ve had (because it’s been an overwhelmingly positive experience, I can’t say enough good things about PASS), then there wouldn’t be any passion. And if there was no passion, there would be no brouhaha and hurt feelings and the developing cliques (oh yeah, people are drawing lines like this was a war in the Balkans, apropos on the 100th Anniversary of World War I). But you know what, if there wasn’t any passion for, in, and around this organization, then it wouldn’t be the organization that it is.

It’s a great organization and people are going to be passionate about it. Cope. Passion is going to lead people to saying negative as well as positive things. Deal. People just might say negative things about you. Develop an epidermis.

Look, we should be able to disagree without being disagreeable, but passion leads us down dark roads sometimes. Let’s try to be understanding of that fact and recognize that the passion that makes this organization great is also the one that’s going to lead to conflict sometimes. Let’s just try to remember that and maybe we’ll be able to work towards sharing the great things this organization does with others and fight with each other less. Maybe.

NOTE: I made an edit about the work I did on the selection committee. It was on a track that I didn’t submit for. There was no way my work there could influence my selection. Plus the fact that the abstract evals and speaker evals were done by two different teams of people. Just want to be clear about that.

Mar 21 2014

PASS DBA Virtual Chapter Talk

I almost forgot to tell you about the Database Administration Virtual Chapter meeting next week, March 26th, 2014. I’ll be doing a talk about query tuning in Windows Azure SQL Database. It’s a talk I’ve given before (it was in the top 10 at the PASS Summit last year). Come find out why you’ll need to tune queries in WASD, the tools you get, and the glorious fact that you’ll actually be actively saving your business money by tuning queries! Click here now to register.

Mar 17 2014

SQL Saturday: You can’t have it all.

SQL Saturday’s are awesome! Let’s get that clear up front. The organizers of SQL Saturday events are glorious individuals. Let’s get that clear too.

I want to be up front about those things because, well, I’m going to be critical.

First though, I want to establish my bona fides for what I’m about to say. I helped organize two SQL Saturday events and two other local events before those. I also help Red Gate Software run half-day seminars all over the country. So, I have some idea what goes into the organizational side of these things. I’ve presented at eleven SQL Saturday events in just the last year. I’m on the schedule for, I think, 6 more between now and August. So, I think I have some idea what it’s like to be a speaker at the events. And, I work for a vendor who puts money and swag up at the events in order to get some advertising. Which gives me some ideas behind what makes the vendors happy too. Further, I’m one of the community. I attend the sessions, talk to the sponsors, take part in the after events, the whole magilla. I don’t think any of this makes me an expert or makes my voice more important than anyone else, but it all comes together to show that I’m not utterly clueless in my opinions (which, I know the adage, opinions are like certain body parts, everyone has one, and they all stink).

Organizers, I’ve seen this issue a lot and it’s just getting worse. This issue is going to hurt you with, in no particular order; speakers, sponsors and the community. What are you doing? You’re trying to have it all.

You want sponsors, right? The sponsors frequently ask for one thing… please, please, please, let us do a presentation so we can show off how wicked awesome our products are to the most motivated people in the region (yeah, the people who are giving up a Saturday to learn technology for their jobs are the best people in the area where that SQL Saturday is taking place). So, you agree to letting the sponsors have a talk… ooh, but when to schedule it?

You want BIG NAME speakers, right? Although I’m absolutely convinced that big name speakers don’t really draw people to your event. Good sessions, usually defined by good session titles, draw people to your event (and good communication on your part through various venues and… well, that’s a different discussion I’ll leave for Karla). But, the belief is there, so people try to get Brent Kline and Kendal Ford and Jes Misner to come speak at their event. BUT, you also want to meet the needs of the local community so you can grow new speakers, so you’re going to take in a bunch of new people too… ah, but how do you schedule that?

And you’re committed to your community too, right? And one of the best ways to show your commitment to your community is to host a panel at lunch. The most common panel is Women in Technology, but I’ve also seen or heard about panels on educating young people, charities, user groups, and all sorts of things. Great stuff really and a big part of why the SQL Family is so wonderful. We really do try to help each other out. We really do care, and those panels give people a chance to communicate what they’ve done to others who may want to contribute in the same way. Ah… but when can we schedule this panel?

By now, I’ll bet many of you know what I’m about to say. But, before I say it, let me point out one more thing. SQL Saturday’s are all day affairs. And if anyone goes to the entire thing, they’re in the building from 8AM to 4PM (or so), so, we’re going to feed them something at mid-day. That really bites into our schedule too.

When can we put all this together? A WIT panel, sponsor talks, new speakers, experienced speakers and lunch…. Hey, hold on. Let’s put it all at lunch. That’s just a gaping hole in the schedule begging to be filled.

And there lies the problem. Putting all this together, all at the same time, hurts something. And, putting it all at lunch, pretty much hurts all of it. It’s hard to get your food and then find your way to a room to eat it in, or, conversely attend the session you want and get your food later, or, try to eat and then go into a session half way through. You can’t do it all. And then, when you think about the audience mix you just created, you’re hurting new speakers because people may skip their session to attend the sponsor session or the WIT panel. The WIT panel is going to suffer because you scheduled an experienced, known, speaker at lunch because you just ran out of room to put them anywhere else. And the sponsors… I’ll be blunt. We want eyeballs. And you just gave them alternatives, and we know they already have alternatives with our competitors doing a session at the same time, but did you have to clean out everyone for the WIT panel too?

In short, organizers, you need to start to pare it down. Don’t try to do it all. You want to support sponsors at lunch? Cool, do that. Schedule the WIT panel to 1/2 hour before the prize drawing (I’ve seen that done, it worked well). You want to have sessions at lunch? Fine. Don’t schedule the sponsors for then. Extend the day and have sponsor sessions before or after lunch. Want to get eyeballs to the local speaker or the big name speaker? Cool, but leave the sponsors out of it. Can’t work out how to fit ALL this in? Then don’t. Don’t even try. Give up on some of it. Pick and choose to make your event yours. But don’t try to cram so much stuff in that you basically make it difficult for the speakers and the community and the sponsors and the attendees.

Oct 17 2013

PASS Summit 2013: Women In Technology Luncheon

This year I was invited to attend the Women In Technology luncheon as a blogger. So I’ll be live-blogging it through it in the same way as I did the keynote.

The WIT lunches are a fascinating, and let’s face it, unique PASS-style event, that have been taking place for years at the PASS Summit. It’s about growth and empowerment for women within technology. But, it’s not some crazy man-bashing event. It’s just another, special, way to network (that thing that PASS does so well).

Panelists are Cindy Gross, Gail Shaw, Kevin Kline, Rob Farley and Erin Stellato.

Mickey Steuwe is acting as moderator.

The theme is Beyond Stereotypes: Equality, Gender Neutrality, and Valuing Diversity is the theme.

The first question: “Do you have to make an effort to fit in?”

It goes to Gail, and in true Gail-fashion, she says, “No I bloody well don’t because I couldn’t be bothered.” She’s awesome.

“Have you experienced more subtle cultural forms of prejudice?”

Rob Farley gets this one. So he goes on to talk about his Christianity as a weird way he’s been affected. For example, a friend wouldn’t come out as gay because he knew Rob was Christian. Interesting point of view.

“Outside of the obvious reasons, how can you tell you’re being treated differently?”

Cindy Gross tells us how she figures that out by talking to friends to get their point of view. But, she correctly says, she has friends who’ll say that “Of course you’re treated differently as a woman” or “Of course it’s not about you being a woman” as knee-jerk positions. She suggests keeping a good thing in mind, of course we’re all prejudiced. Not badly, but on purpose. You assume that a person who is an adult will act like an adult and a child will act like a child. These assumptions are prejudices and judgements we make every day because you can’t start from a clean slate.

Kevin adds that the concepts of stereotypes are, at least in part, based on reality because they act as short-cuts. Indexes if you will. It was pretty cool. He also went on to talk about the differences between introverts and extroverts.

Ooops. Too much typing. I missed the question.

Kevin is talking about how we, as DBAs or data pros, frequently will spend more time trying to figure out how to tune indexes, etc., instead of trying to figure out our co-workers. I don’t disagree with that at all. I actually did get a set of tools for doing this at my previous job. They were occasionally crazy to work for, but they really did think through how work is supposed to be done and done right. Kevin goes on to discussing the interesting differences between ethical and moral.

12:36

Rob adds to this, in a nutshell, if you see something, say something. And, the fact is, this is true. We, all of us, need to be willing, and able, to stand up and make noise if we think things are wrong. It’s a must.

However, I’m not so sure about where the discussion went then. A piece of advice was given that said “If your friends are trying to get you to change who you are, maybe they are not your friends.” Well…. Can I use a DBA phrase here? It depends. What if they’re trying to get you to change for real and positive reasons? You’re doing drugs, smoking, terribly overweight, out of shape, in a destructive relationship… Anyway.

Cindy then talks about how, because she advocates for people of various styles, she has been associated with them. In short, because she supported gay people, others thought she was gay.

Erin is FINALLY asked a question.

She talks about how, regardless of who you’re working with, you need to find some type of common ground with others. That you can do that, even though it may be somewhat painful or difficult. But, once you’ve done it you really can make a difference in how you work with people.  Mainly because, you have to work with people. It just makes sense.

So, Erin not only got a question, but she knocked the answer out of the park.

12:50

next up, how to get things working with teams. Oh nuts. Good luck. This is a horribly hard situation for ANYONE in ANY situation. Male/female or other negative or unhelpful prejudices. Cindy Gross also did racing on dirt bikes in Texas. She tells the story of getting a bunch of girls together, calling them dirt chicks, and giving them popcicles at the end of the race. Why? Because in school the girls who raced would get ostracized at school because they were beat up from the races (and yes, dirt bike riding can mess you up good, it’s a blast).

I have to leave early for another event.

Thanks PASS, panelists and Mickey. Great Job!

Oct 17 2013

PASS Summit 2013 Day 2 Key Note

KILT DAY!

Today we have to eat our vegetables and then get lots and lots of sweet desert.

Or.

Today we hear about PASS Finances as a part of the official annual meeting and then we get to hear Dr. David Dewitt speak (completely and utterly getting our nerd on and squeeing like teenage girls at a Bieber concert).

I will be live-blogging this event, so watch this space.

8:20: Douglas McDowell kicks off the key note today.  the vast majority of the money that runs PASS comes from the Summit. That’s right, by attending the Summit you’re also supporting the organization. The Business Analytics Conference, which kicked off this year also provides quite a bit more money to the organization.

8:25: PASS has changed its budgeting process. At this point, there is about 1 million dollars (American) in the bank. That means they’ve got a cushion should an event go south. That’s very important.

The amount of money spent on the community last year was $7.6 million. 30% of that is focused specifically for international audiences (which begs the question, how much money comes FROM the international audiences). The money is spent on Summit, BA Conferences, Chapters, SQL Saturday, 24 Hours of PASS and 520 web sites (woof).

8:31: Bill Graziano, PASS President, takes the stage to say goodbye to PASS Board members leaving the board. Douglas McDowell, who was just talking, is leaving the board after six years and being a volunteer since 2001. Rob Farley is also leaving the board. Rushabh Mehta comes on stage after eight years on the board. He’s the Immediate Past President, a role that automatically rolls off the board after a couple of years.

Next up, Thomas LaRock, the currrent vice-president of Marketing and the incoming PASS President. We had about 3000 unique viewers online at the live PASS TV (which I launched this morning, talking about KILT DAY!). The new board positions are Adam Jorgensen, Executive Vice President, Denise Mcinerney Vice President Marketing. Jen Stirrup, Tim Ford and Amy Lewis are coming onto the board.

In 1999, the Summit started. That’s 14 years. I’ve made 9 of them in a row.

8:38: PASS Summit 2014 will be in November 4-7 in Seattle next year. The PASS BA Conference will be in San Jose, CA May 7-9 in 2014.

Remember there are tons of networking opportunities.

8:41: What, Why, How Hekaton with Dr. David DeWitt

Let’s get our nerd on.

Dr. DeWitt is one of the things that makes the Summit.

WHAT:
Hekaton is memory optimized but durable, very high performance OLTP engine, fully integrated into SQL Server 2014, Architected for modern CPUs. It really is a wicked cool technology. I still don’t by the concept that you don’t need new hardware for this, but that’s not questioning the utility of the functionality.

WHY:
OLTP performance has started to plateau with current technology. The increases in CPU just aren’t going fast enough any more, so they have to find something to figure out how to improve performance. The goal for Hekaton was a 100x improvement. They didn’t make that, but they got between 10x and 30x improvement, which is pretty amazing.

You can’t just pin all tables in performance. Latches for shared data structures are going to hurt. they hit locks for control mechanisms and the execution plans generated won’t be improved.

The implications of a buffer pool are that you get storage over time.

You’ll need to track down the slides to understand some of what I’m saying in this live blogging. It won’t make sense without them.

I found it here

So a query needs a page. It checks for the page. The query gets blocked until the page gets allocated and then it continues from there. But, another query can be blocked by the process coming in. So, they added latches to the pages in the buffer pool. He shows how the latches allow multiple queries to find objects in the pool, but mark them as being used. But this ultimately runs into performance because the shared data structures need latches and they consume time to maintain.

8:55:

You also have to have concurrency control, in short, locking and blocking (you know, the stuff that NOLOCK “fixes”). Jim Gray, mentor to Dr. DeWitt, came up with two phase locking. So a query gets the lock type from the manager and then when a query releases locks, they can be reused. This basically sets up the idea of serial locking to get things done correctly.

When the database lives on disk, the processing time to get a query and create a plan, can be trivial (not always), but if the data is in memory, that becomes way to expensive.

All this is the reason you can’t pin stuff in memory.

Shared data structures have latches. Concurrency control uses two-phase locking. Query plans are through interpretation.

Hekaton, on the other hand, uses Lock-free data structures, meaning no latches. They’ve stopped using locking for concurrency control. They use versions with timestamps + optimistic concurrency control for Hekaton. And queries are actually, literally, compiled into a DLL. That’s right. COMPILED. Queries have been “compiled” into an interpretation layer all this time. Not literally compiled. But, with this, they’re getting turned into DLLs.

There are now three query engines in SQL Server. Relational, Column Store and Hekaton. These are three distinct stacks. Queries can span all three.

9:06: First, you create a “memory optimized” table. That table does have limits (look them up) in structure and data types supported

Second, populate the table, but, you have to make sure that data will absolutely fit in memory. You can put 5gb of data into a system with 2gb of memory. NO PAGING TO DISK. It’s in-memory, right?

Third, run queries, but there are some language restrictions.

9:12: HOW:

Lock Free Data structures. the data structures are truly rocket science. They make query optimization look simple (OW!). These were invented by Maurice Herlihy at Brown University. It’s not really lock-free, but since it’s not about concurrency, it’s about being latch-free. Dr. DeWitt tells us he could explain it in about 30 minutes, but instead we get a demo.

he’s showing that latches slow down more and more as the number of threads hit the system. Yet the lock-free approach actually increases. Then, when updates occur, everything stops until the update completes. The lock-free mechanism doesn’t stop at all. It doesn’t even slow. The lock-free mechanisms took 5 years alone.

Multi version, optimistic, time-stamped concurrency control: The assumption is that conflicts are rare. Transactions are run to “completion” with no locks. Then conflicts are resolved later. Multiversion means that updates create a new version of the row. Each row version has a time range. Transactions use ther being timestamp too select correct version. Timestamps are used to create a total order for transactions to obtain equivalent of a serial order. This reduces the number of threads and that reduction rediuces the likelihood of locking.

Read committed versions start. Updates create new “tentative” vversions and then the DB tracks the rows read, written and scanned. Then updates go through a pre-commit step which gives you validation and then the concurrency control goes through it’s work in post processing.

Timestamps are just counters. So you get begin and end times so you know how to track mechanisms. End times are always unique and that’s how you can manage who goes first in terms of concerency.

So a row gets tagged with a begginning ts and then when it completes a unique end time time stamp. When it starts, you get a new version of the row, with pointers linking the versions of the row. There will be a “magic identifier” assigned from the transaction to the versions of the row. An end time stamp to the older row and now end at all, but a begginning time stampe on the second row. So, this means no latches were used and there were no locks set and there were no blocks of other transactions. This creates the basis of multiversion concurrency control.

So if you have two transactions running concurrently, you’ll see the first transaction create a version of a row with copies and versions. Then a second transaction tries to read the row. If it’s timestamp of the second version which was earlier than the first transaction, it’ll use the older version, because the time stamp of the end time of the second transaction must be later than the current time, because it’s not complete yet.

Yeah, that sounds confusing, but looking at the slides you’ll get it.

Then, a clean up process has to occur. When the begin time stamp of the oldest version in the system ticks past a more recent version, then the older version will get removed. This clean up is cooperative, non-blocking, incremental, parallel and self-throttling.

Each version contains a valid time stamp range. You get transactions through time stamps and versions. Then a transaction will read only versions of rows that valid when time overalps the beginning of the range for a transaction.

THEN, we have to go through Validation.

1. Transaction obtains a unique end time stamp
2. Determine if the transaction can be safely committted
3. Validation steps depend on isolation level (and check the slides for details).

Each version read is checked to see if they’re still “visible” or “valid” at the end of a transaction. This also helps with phantom avoidance. But, everything is in memory and we’re not getting locks, so while expensive, it’s actually still cheaper than the old versions of latching and locking.

Post-processing goes through three phases. You get a log record with all versions of the row and the primary keys of all deleted rows. A single i/o is written to the log. For all rows in the transaction writeset the transaction id is replaced with the end time stamp.

I sort of understand all this. The trick will be to remember it and then learn how to explain it to others.

But, you have to have checkpoints and recovery. Data is stored in the logs during checkpoint operations, roughly the same as normal. Recovery loads the know checkpoints and scans logs to recover all work since then. It has full integration with High Availability.

9:43

Then we got to queries and quer plans. You can run regular queries in what they call interop but you sacrifice performance. Instead, you want to compile it. You get physical plans, kind of the same way as you used to (not quite the same, but I was hitting a snag when he explained that part, check the slides), but then it goes through a translator which generates c code. Evidently, really ugly c code. But then the compilers is called and then you get a DLL. This is 100% totally specific with no functions. Then you get a DLL loaded and invoked. You never recompile that query again.

The number of instructions is interesting. A classic table can take 700 instructions to find a row. With Hekaton, 332 and with a native qp, 75. Even if you don’t find a row, it’s 300, 110 and 31.

Interop can get up to 3x improvement, but there are still language limits. Same issues with native mode, but you 10-30x improvements with that.

Finally, he’s going through a whole bunch of performance improvements by various, REAL, companies using it now.

The whole thing is that memory prices have been declining. We’re seeing lots of CPU cores designed for concurrency, but we’re still hurting from CPUs through the lack of compiled code. But, it’s supported by the hardware trends.

Oct 16 2013

PASS Summit 2013 Day 1 Keynote

I am liveblogging the keynote from the bloggers table at the PASS Summit again this year. Just keep scrolling.

Watching the introduction video as people trickle in. All the other bloggers are setting up. I get in early. I didn’t rearrange the seats this year. I see others doing it now.

8:11: Watching the videos of all the attendees registering and meeting people at the start of the event and last night’s welcome reception is awesome and fun.

8:21: The lights go down and the videos of what everyone is looking forward to at the Summit. In keeping with our location, right next to the NASCAR Hall of Fame, we’ve got a bit of a race theme going on. We’re seeing current PASS President, Bill Graziano having a dream about driving a car. He’s starting off with the list of PASS Board members, just so you know who it is that’s doing the most of the work for this fantastic volunteer run organization.

We’re also getting a listing of the 700K hours of training that have been put together by the PASS Organization.

8:30: We get to find out who the PASSion award winner is. Amy Lewis, who is absolutely an amazing person, is the winner this year. Ryan Adams, sitting right behind me blogging away, was an Honorable Mention Volunteer. Well done Ryan. They also ask you to nominate outstanding volunteers for the year too. Make sure you do. This really is a volunteer run organization, so you need to support the volunteers.

8:37: Quentin Clark takes the stage with a listing of companies that have adopted Azure technologies. If you read my blog, you know it’s one of my passions (although I’m still a query tuning freak). You need to get going with it.

Quentin is starting with the concept of “A story about transformation.” He’s showing how Brick and Mortar and Internet are helping each other, not hurting each other. Integration between the stores and the internet made things better. The comparison is of course aimed at telling the story between on-premises computing and cloud computing. It’s a compelling story. We’re seeing how they’re rolling out a series of software that is available now, or in the very near future, which is different than past key notes where we saw stuff that was coming out “next year” or “real soon.” That’s an awesome approach.

8:49: We’re seeing all kinds of new technology in 2014. They’re not fundamental changes causing a rewrite of technology. Instead they’re additional technologies, updateable column store index and in-memory tables and indexes for OLTP. It’s awesome. It makes it possible to do more, when you need to, rather than only after rewriting your entire app. I think the work they’re doing in Azure is making it possible for them to release more frequently to the on-premises versions without causing breaking changes. It’s a great way to get things done.

8:54: I love the demos when they are more realistic. We can see a 10% improvement on queries, just by using Memory Optimization. They’re also introducing the Native Compilation, which means a true compilation, turning a proc into code on the structure of the SQL Server instance, not simply a query plan stored and accessed in cache. That resulted in another 11X improvement in performance. The issues around this though is that most of this technology is very hardware intensive. You’ll have to have big boxes for this to really help you. So yes, we’re getting great new technology, but you’re only going to be able to really blow it out of the water with other great new technology.

The main points they want to tell us is that it’s built into SQL, which is 100% true. They also want us to know that there is no new hardware needed, which doesn’t make sense. You can’t put stuff in memory without using more memory. It has to impact existing hardware. However, I see the utility of it.

8:59: They’re expanding on the abilities for availability and recovery. Sorry people, but this means taking advantage of additional functionality in Windows Azure. But it works. I’ve seen it in action in production environments. You can set up AlwaysOn secondaries in Windows Azure. You can backup to Windows Azure. It’s not a requirement to migrate your systems out of your environment, but to use the Azure system as your backup and recovery mechanisms.

9:05: They new backup tools are great. From 2014 we get backups that are encrypted without requiring the database to be encrypted. That’s great. They’re also making it so that you get backups that you can automate based on data changes, not just a timing thing. That opens up whole new ways to protect your systems. I’m excited by this stuff. I’m also interested to see that they’re releasing a tool that will let you incorporate backups to the cloud from your 2005, 2008 and 2008 R2 systems, not just 2012 and 2014. That’s great, but it hurts companies like Red Gate that have been offering this as a product to people for years. Ah well.

9:17: Microsoft continues to expand it’s Hadoop offerings supporting it through the desktop and through Azure in HDInsight. Most of this stuff is in preview, but they have people using it in their production environments, so it must be relatively solid. The point is being able to query everything. Not simply this type of query from structured data and this type from nonstructured.

9:28: Mostly talking about BI stuff. I’m glad we’re serving out the data in better and more interesting ways. I just can’t get too excited about it. In the mean time I’m actively configuring a CTP2 of SQL Server 2014 in an Azure VM while the event is going on. People are trying to download it instead of setting up a VM. They’re crazy.

9:45: That was a good key note.