blog

Posts by Elliot Noss

Help Save MySQL

Help Save MySQLOne of our core values at Tucows and OpenSRS is that the Internet is the greatest agent for positive change the world has ever seen. And we strongly believe that open source tools are central to the continued growth and health of the Internet.

You may have heard that Oracle has acquired Sun, and along with it, MySQL, which is is a central building block in the suite of open source tools.

While MySQL holds that position in the world of open source software, it is not important to the Oracle acquisition of Sun. In fact, all of the most important reasons for Oracle doing the acquisition would still be in place if Sun had no role whatsoever with MySQL.

The “cost” to Oracle of freeing MySQL is very low. The benefit to the world is extremely high.

With that in mind, please consider lending your support to the campaign. We’ve added our support by signing the petition. The Save MySQL website has lots of information about why the community feels that MySQL is important, and what you can do to ensure it stays open and freely available to the entire Internet community.

Letter to OpenSRS Email Resellers

Dear Customers -

On behalf of all of the members of the OpenSRS team, please accept our sincere and deepest apologies for the service disruption on Cluster A this past weekend.

Many of you have asked, “How could we have let this happen again?” We initially were led to believe that we had a software problem. We have now determined that the string of service problems on Cluster A are related to a hardware problem inside one of our NetApp devices.

Below is a letter of explanation I received from Jeff Goldstein, General Manager at NetApp Canada.

We are not without fault in this situation. Network-attached storage is complex and we trusted our vendor to provide us with accurate advice related to our problems. In hindsight, we should have pressed earlier for replacement hardware.

Please rest assured that we are dedicated to providing a reliable email service and will be working tirelessly to restore your confidence in us. An incident report is available at OpenSRS Status.

Sincerely,
Elliot Noss,
President and CEO, Tucows

Dear Elliot Noss,

I am writing today regarding the recent outage that occurred this past weekend with Cluster A of the OpenSRS Email Service.

As you are aware, Cluster A of the OpenSRS Email Service has experienced a number of service degradations related to issues with our NetApp storage device. Our engineers here at
NetApp worked closely with the technical operations and development teams at OpenSRS to trouble-shoot and resolve these issues. In each of the cases, we believed a software
fault was the cause.

The intermittent problem turned out to be due to the hardware shelf controller as well as firmware in one of our NetApp storage devices, which caused the issues on Cluster A.

We are deeply sorry for the inconvenience that resulted from these hardware and email service issues.

One of the promises we make to our customers is that our solutions provide highly available data management and in this case we let you down.

To begin to resolve this issue, we’re taking immediate action to replace the hardware and firmware in Cluster A at our expense. Our engineers will then test and evaluate the components involved to determine what specifically went wrong and apply those findings back into our own quality control
teams.

Our two companies have been working together for the past nine years. We value our relationship and will work hard to restore your confidence in NetApp and our solutions.

Again, please accept our sincere apologies.

Regards,

Jeff Goldstein
Canadian General Manager
NetApp Canada

Open Letter To Our Email Service Resellers

Dear Resellers,

I am writing today to speak to you directly about what happened this week with Cluster A of our Email Service. This will not refer to specific elements of the outage, there are other venues for that. The things I most want to communicate are my deep sorrow, why it won’t happen again and what we will do for you.

More than anything one thought keeps going through my mind as I think about this, the future determines the past. I will return to this thought.

First, and most importantly, we are sorry. I am sorry. I have been in this business a long time and do not know if I have ever been more sad about what we have done to you, to your customers and to how people think about us. An email outage in 1995 was different from one in 2000 and even more different from one in 2008. I know what this does to your reputations, to your customers and to your staff – and I and so many people here are just sad about that.

While it seems trite right now, we really define ourselves by how we make it easier for you in your businesses and with your customers and in our deep understanding of those relationships. That means the pain here is that much greater and believe me I know our pain here does not matter, yours does. Just know we are grieving.

Second, what will we do about it and why will this never happen again? I know for some of you that doesn’t matter, you are done with us, but I want to express this for the rest of you. Let me start here with things that were not the problem, old equipment, people, capacity or redundancy. The equipment is new, the people are great, we have plenty of capacity and redundancy. What this will mean for us is clearly the need to take the other elements of the service to a completely new level. Here I mean monitoring, change management, emergency protocols and procedures and operating efficiencies.

We had decided long before this that the most important part of email was reliability, not features, not groupware, not web 2.0 integration but reliability and deliverability. I have been at this a long time and really believe that these people and this service can be the best in the world, better than Google, Yahoo or Microsoft and most importantly the best partner for service providers. We owe you this and will deliver it.

Lastly, what we will do for you as a result of this? Let me start here by saying two things, we will certainly be doing something and that there is nothing we can do that will make up for your loss of reputation in your customers’ eyes. We know that. The people who will participate in that decision are fried right now, as I know even in your anger you can well imagine. I will ask your indulgence that you give us this week to make our plan in this regard.

There is one thing that I can offer now. I would like to make myself personally available to any of you who would like me to either reach out to your customers, or to any specific customer, with a letter, an email or a phone call. I know this will not often matter but perhaps in a few cases it might. My message here would be simple, this was our fault not yours and while you are responsible for the suppliers you pick, you had good reason to pick us and it was us who let you down. This offer stands whether you are leaving or staying.

In closing, the future determines the past. If we move forward and run the most reliable, service-provider focused, email service the world has ever seen this will be remembered as the few days that turned it around, as being a very important event in forging out mutual future. If we have no change in reliability or in service levels this will barely be remembered. It will just be a point on a mediocre line. I will do everything in my power to make it the former not the latter.

Regards,

Elliot Noss

My hand hurts, I’ll cut off my arm

Yesterday a large webhosting company, Dreamhost, told the world that, while they would continue to provide email, their email service was not that great and suggested their customers should probably use Google’s Gmail instead.

They provided some fascinating data about email and support costs. My two favorite nuggets:

“Just over HALF of all the support requests we get are about email. Everything else we offer, combined, doesn’t add up to the amount of trouble, expense, use, and effort that goes into “simple” old email.”

and:

“If a web server with maybe 750 customer sites on it were to go down for even as long as five hours, we’d probably get two angry messages about it. But if email goes down for the same number of customers for just five minutes we’ll have already received 50!”

And they are clear as to their view of quality:

“(email is) something the big free email providers like Yahoo, Microsoft, and Google can do better.”

This post was picked up on Slashdot where the discussion, not surprisingly, swung back and forth between “I am a sysadmin managing 20 domains and use Google Apps and Gmail and love it” and, “You should always run your own mail server for privacy purposes and, well, its just plain fun.”

Both the original Dreamhost blog post and the resulting Slashdot discussion completely missed the point. Luckily the comments on the Dreamhost blog did not. They were very clear.

Overwhelmingly commenters said that they often came to Dreamhost for hosted email, they did not trust or want to use Gmail for their business email and many of them would immediately leave if Dreamhost discontinued offering email.

Every service provider should be required to read the Dreamhost blog post and, more importantly, the comments.

Whether geeks like it or not, the vast majority of people want and need simple, reliable email that is easy to use AND they want a supplier who will help them use it. That means providing phone support as well as resources to make things simpler. Support data provides golden information for i) how a service can be improved and ii) what your customer’s needs and wants are. Guess what? People are willing to pay for this.

Contrast the Dreamhost view with that of Rackspace. Faced with, I suspect, the same or similar data, Rackspace responded by going out and buying Webmail.us.

It is amazing to me that because most service providers have chosen to give away email they take that as an existence proof that people do not or will not pay for a quality email experience. People will pay over $80/month for a single cup of coffee per day. People paid Geek Squad over $1 billion last year to “set up” their wireless routers. Every geek knows how hard (or not) that is! My ten-year-old son does just that for my mother-in-law. With regards to email specifically, RIM, the Blackberry people, have a market cap of over $75b JUST FROM PROVIDING A PORTION of peoples email needs!

People, especially small businesses, use email more than anything else on the Internet—much more than they use or need web hosting. Service providers are in the business of making the Internet easier and more effective—whether they like it or not.

Geeks who run service providers may find Gmail great. Human beings, not so much.

Thoughts on the Domain Name Price Increases

I wanted to share some thoughts with all of you on a dark day in Internet history. On October 15th the price of a .com will increase by $0.42, marking the first price increase in the history of the modern Internet. Worse, this now signals a near-annual event that will take place in all major gTLDs. It is simply wrong. My full comments in the public forum in Puerto Rico in June are here.

While I do think Verisign has shown a lack of stewardship of this key public resource, I lay the primary blame for this on ICANN staff who put this forward and on ICANN board members who voted for this (it should be noted that the vote was 9-5. One of the closest in ICANN annals). As I said in Puerto Rico, shame on you. We all, all of us involved in the ICANN process in any way, owe the Internet public because of this.

It is important that we do not use this as a sign that ICANN, the idea, is failing. We should not confuse bad execution with bad strategy. The role of ICANN as an example of truly global, not International, governance is important. The role of ICANN in keeping the Internet free from government control and by that the predation of special interests is vital.

And it is a challenging environment. There is a debate inside the Registrar constituency right now, effectively re-fighting a battle that was already won, but sloppily implemented by staff. Many of you (the “you” here is our customers) will have already dealt with the end-user problems created by Go Daddy and Network Solutions in their “interpretation” of transfers policy in the name of “security”. For me this is simply deja vu.

Service providers, there is something you can do. Something important. There has been a process of GNSO reform going on inside of ICANN for the last 18 months. The GNSO is the primary policy-making body in the ICANN process. They are the ones charged with making policy for gTLDs. The board only has the power to ratify policy. Staff only has power to enforce and interpret policy.

Inside of the GNSO there has been something of a stalemate for the last few years. One of the chief reasons is that the Internet Service Providers Constituency (”ISPC”) has consistently sided with the Intellectual Property Constituency (”IPC”) on things like whois access and new gTLDs. I have been in and around the ISP industry now for 13 years and the ISPC does not look like any ISP assembly that I know.

I have been advocating change in the GNSO reform discussions. In Lisbon in March and again in Puerto Rico in June I have advocated a recasting of the ISPC. My position is that it should be a constituency for companies who stand between the “contracting parties” (ICANN-speak for Registrars) and end users. Most of the industry calls these people resellers (an old OpenSRS anachronism). They have no place or voice in the ICANN process right now and they need one. We have been trying to advocate their interests (your interests) for years. You can do a better job of it than we can.

When it comes to transfers, to whois and to most issues of DNS policy they (YOU!) are a voice that needs to be heard.

My advocating is the easy part of the battle. The harder part will be to actually have some of you folks do it. So take this as a plea to storm the ramparts! Now! In the next couple days we will post more about the ISPC, what can be done, and how to do it here. The time commitment is VERY small and the impact can be very large. Just ask George Kirikos what a little effort can accomplish in the ICANN process!

Questions to Ask Before You Pick Your Domain Name Registrar

By Elliot Noss, President & CEO, Tucows

Reglogo

The outcry over RegisterFly has generated a lot of discussion about the regulation and oversight of registrars. (A domain name registrar is a company accredited by the Internet Corporation for Assigned Names and Numbers (ICANN) to register Internet domain names. Tucows is an accredited registrar.) In our view, much of the furor and finger pointing is missing the point. The interests of the businesses and individuals who buy domain names have been lost in this debate. There are a whole raft of existing rules and policies, defined by ICANN, that govern how the domain name registration system should work. The problem is that the people who buy domain names (formally known as registrants) are not familiar with the rules and do not know how to help themselves.

The bad news in this is that fixing how registrars are regulated and, more importantly, how the existing rules are enforced, will take some more work, especially on the part of ICANN. In the meantime, registrants need to change the way they think about who supplies them Internet services. They will need to stop focusing on price and start focusing on customer service, the services offered and increase their awareness of the rights that the existing rules give them.

Domain names are valuable intangible assets and are central to a registrant's use of the Internet. We wanted to try and help folks out by providing a set of guidelines that are intended to help registrants make good decisions for themselves. This guide is not intended to be exhaustive, but is written as if a close friend has asked us for advice. Here are the 10 questions everyone should ask a registrar before buying a domain name from them. We don’t expect registrants to necessarily dig in to all of them, but it is a place to start.

1. What is your primary business model?

This is an interesting one and needs a bit of context to explain. All major registrars used to earn money in a very similar way. They were mostly in the business of selling retail Internet services. A few, Tucows included, provided mostly wholesale Internet services to Web hosting companies and ISPs. Over time, some started to specialize in catering to large corporations who registered domains in many different countries and to other, narrower, segments. With the massive growth in direct navigation (when Internet users type their desired destination directly into the browser address bar) and the secondary market (domain owners buying and selling domains directly) things have changed. Many registrars now make more money from ads on domains in their portfolios, auctioning names and selling names in this secondary market than they do from registration fees for new domain names. Some make much more from those sources.

That is not bad in and of itself, but it might influence how they implement services like a redemption grace period.

Recommendation: Understand what business your registrar is in, and also understand who owns them and what business they are in.

2. Do you make transfers as easy as the rules allow?

Domain name portability (or “registrar transfers”) has been a source of contention in the domain registration industry from the onset of competition. For years large incumbents, at the time primarily Network Solutions the old monopoly, made it difficult for registrants to transfer their registrations to competing registrars. Since 1999, Tucows has spent thousands of people hours in both customer service and at a policy level to effect change. As a result, domain name transfer policy is now much more proscribed, much more predictable and much easier to effect.

Unfortunately, some registrars still make transferring a domain difficult. An example would be by requiring 60 days following a change of ANY contact data before a transfer can be made (another is discussed in 3 below). Thus, if you want to move to a new registrar and need to change the administrative contact email address in order to approve the transfer and do it 55 days before expiry you will not be allowed to transfer for an additional 60 days. Of course at that point your domain expired five days ago and you are at the mercy of the registrars grace period policy which often does not allow transfers only renewal of names.

Policies like this have the effect of confusing registrants and in some cases even result in people losing their names (especially if combined with short grace periods and quick triggers for auctions).

As a customer, I tend to look favorably on companies who make it easy to change services. I take that as having confidence in their ability to keep me as a customer on merit alone.

Recommendation: Understand all the rules and restrictions concerning transfers that your registrar employs. If you find them too restrictive perhaps it is a sign to keep shopping for another registrar.

3. Do you allow for easy locking/unlocking?

Combination-Lock

Domain names can be locked to ensure no changes are made to either the contact information or to the settings. This is a useful feature, however many registrars have used the locking function as a form of lock-in! They do this by locking ALL names and making the ability to unlock obscure and/or difficult. If combined with long customer service queues, limited documentation, and poor email support a domain locked by a registrar can have the effect of making changes and transfers nearly impossible.

Recommendation: Make sure you know whether your registrar locks names and if they do, how to unlock them.

4. Do you make it easy to opt-out of auto-renewals?

Auto-renew can be very useful. Well-intended registrars provide auto-renew functionality to give registrants peace-of-mind. Just “set it and forget it”. This works when the registrant is in control.

However, some registrars set all their accounts to auto-renew by default. In certain situations like monthly billing, this may be the right choice for a registrant. The problem with auto-renew comes when you don't want to renew the domain name or you want to change registrars or when it is set too early in the renewal process.

Recommendation: Find out if your registrar offers auto-renew, what their default setting is, and how to change it if you want.

5. Do you tie domains to your services?

Some registrars use domain names as a loss leader to upsell other services. There’s nothing wrong with that as long as you can easily change any and all relevant information to what you want it to be. If you want to change where your nameservers point you should be able to change them. If you want to re-point the mx record controlling your email, you should be able to.

Most importantly, does the Registrar let you change the contact information on your whois record to whatever you want? With the exception of possibly the technical contact there is no excuse for not letting you change the information on record to whatever you choose. And remember, in many respects the administrative contact is the most important contact in the mix. Make sure you set it to who you want it to be.

Historically, many web hosting companies and ISPs set all the contact information to their own and handled all of the administration for generally confused registrants. Many still do and overwhelmingly they do so appropriately. Others, however, do not.

Recommendation: Ask about and understand any restrictions on changing contact information.

6. Do you offer Whois privacy? What are your privacy policies in general?

Whois privacy is much discussed with the whole RegisterFly mess. Whois privacy is a service offered by registrars who act as an intermediary to protect the personal information required when registering a domain. And it is generally a good service some registrants will want to use. Does your registrar offer it? If they do, how much do they charge? Many registrars charge more for whois privacy than they charge for the domain name registration itself.

What are their privacy policies, and perhaps more importantly, what is their practice when enquiries are made? Registrars diverge wildly in how they treat incoming requests for information or complaints.

Recommendation: Understand what both the policies and the practices of your registrar are with respect to whois privacy and privacy policy in general.

7. What are your policies on compliance issues like litigation, ownership disputes and WDRP?

There are a number of compliance issues that can effect domain names. Does your registrar have a dedicated compliance department? Will they inform you of a dispute? Will they help you understand what your rights and responsibilities are? Recent examples, have also called into light how registrars respond to court-orders and requests from law enforcement. The practices of registrars vary widely here and are central to your ownership.

Recommendation: Know your registrar’s policies and practices with respect to compliance issues and how you contact the compliance department.

8. How easy is it to contact you?

How easy is it to find the contact information for customer support? Many online businesses focus on making it easy to buy their services but MUCH more difficult to obtain support for them.

Recommendation: Try and find the contact email addresses and phone numbers BEFORE buying.

9. What happens when my domain expires?

Every registrar has a grace period after the actual expiry of a domain name. The timing of when names “drop” is extremely important in figuring out how much room you have to make mistakes. It used to be that almost every registrar offered a 40-45 day grace period and after that an ICANN-mandated 30-day Redemption Grace Period (”RGP”). During RGP the name was more expensive to redeem (usually $100-150) and could only be redeemed by the registrant, but at least you could still get it back.

With the growth in what’s called the domain name aftermarket this has changed significantly. Fewer and fewer domain names are “dropping” or becoming available for re-registration on the open market. These days, grace periods have shortened for many registrars and what happens during this period has changed. Virtually every major registrar now either auctions names when they expire or takes the names they want from the pool of expiring names for their own portfolio (or some combination of both). The important thing to understand is how registrars do it and when do they do it.

Recommendation: Find a registrar that provides at least a 40 day grace period and that respects an actual or implied RGP for 30 days after that. In other words, you should not irrevocably lose your name for at least 70 days after the expiry date.

10. Are you a registrar or reseller?

This is often misunderstood. A registrar has been accredited by ICANN. Resellers purchase domain names from registrars for their customers.

In our view, dealing with a reseller is often preferable to dealing with a registrar directly, especially if you are buying a bundle of services like email or a web hosting package and even if you are just buying a domain name for use with services at a later date. By the way, any small business or individual will be buying a domain name to use on some level with other Internet services making the distinction between reseller and registrar somewhat artificial. What you really need to look for is a quality supplier of Internet services that you can trust and who helps make using Internet services easier.

Recommendation: If using a domain name reseller, simply make sure that you understand all of the above policies as provided by the registrar with whom they choose to do business.

Summary

The domain name business continues to change. Despite evolving business models, we believe registrant rights deserve to be front and center.

Our suggestions:

  • Make sure you know how domain expirations work. Getting a domain back after it has expired is increasingly difficult and costly.
  • Know how to lock and unlock your domain so you can make changes if needed. Also take a look at transfer policies, and make sure they are reasonable and easy to understand.
  • Before you hand over your credit card, understand how a registrar’s auto-renew works. You should be able to stop an auto-renewal if you don’t want to hold on to a domain name.
  • Make sure you’re working with a reputable registrar or reseller who is in the primary business of providing services to businesses or consumers. Make sure you can find their contact information and privacy and compliance policies.
  • Ask people you trust who they use to register domain names and find out if they have ever had to have the supplier help them with problems. Often, “I use xyz and have never had a problem”, means just that, they have never had to DEAL with a problem. It is when problems arise that the difference between a good and bad supplier becomes clear.

By doing your research you may be able to protect a valuable asset – your domain name.