Securing user passwords

Securing user passwords

An application that stores user credentials has a responsibility to secure them; beyond the need to keep integrity of your own application.  No matter how often users are discouraged from reusing passwords, some number of users are still going to do it; and probably with the same user name, if available.  So hacking user names and passwords on your site will compromise the security of some of those people on other sites.

As a web developer you may create a fair number of web sites that require user authentication.  There is no shortage of articles telling you to use hashing, salting and key stretching.  See the references below for a few of them.  This article is going to focus on a .net solution to implement these security concepts.  But please don’t just rely on the code I share here.  Study and understand the concepts presented in the references, especially, Salted Password Hashing – Doing It Right

Disclaimer: I am a web developer, not a cryptographer.

Application Security

As the web developer, it is your job to ensure the application cannot be used to compromise the database.  If you assume your servers have the best security in the world and that hypothetically a hacker cannot get to your database, you still need to make sure your web site cannot be used to hack the database, and more importantly the data belonging to your users.  But not just on your site, but on other sites where they may use the same password.  No matter how wrong it is for the user to reuse passwords, in the public eye you will be to blame if their passwords are compromised.  From an application perspective, you have a few responsibilities here.

I. Prevent dictionary attacks

If your sign in screen, web service calls, or any other access point does not limit the rate that any given name can sign in then the site is open to a dictionary attack.  That means a hacker could attempt sign in many times a second to guess the password of a given user.

My policy for a low security site is generally to allow 3-5 sign in attempts per user name in a 20-30 minute period.  For a legitimate user that usually means they can enter the wrong password 2-4 times.  If they fail again, then they have to wait 20-30 minutes to try again.  For a medium security site I would normally allow 3-5 sign in attempts and then lock them out until they take the required action to unlock it.  That may be a security question challenge, or it may be sending them an unlock code to their primary email address to reset the password.  It depends on your requirements.

II. Don’t reveal if a user exists or not

When a user attempts to sign in and fails, then do not say why it failed.  Do not say ‘user does not exist’.  Use something like ‘Invalid credentials’, or even ‘invalid password’; as long as you say that for non-existent users as well.  But the error message is not the only way to give away if the user name exists or not.  It could be that the forgot password, or forgot user name functionality only allows for registered users.  Or maybe an Ajax call on the page only returns data for registered users.  Anywhere in your application that returns data for users, should always return that data structure for non existent users as well.  Never have an error message for any publicly exposed aspect of your application stating the user does not exist.

In the reset password scenario, you are likely prompting for an email address and then sending a short lived reset link to that email address.  If the email address is not tied to a user account you should still tell the user a reset code has been sent.  Just in case that email address is the person’s real email address, but they don’t have their user account tied to that email, you should still send an email.  Tell them what happened and ask apologize if they did not request the password reset.

Another use case for exposing existing user names is when a person is registering for a new user account on your site.  A user sign in name generally must be unique within the site, so there must be some kind of error or warning if the name chosen is already in use.  So it seems that could be a backdoor into finding what user accounts exist.  However, you can seed your database with deactivated user accounts that also cannot be reused by a new registering user.  Then an error message stating the name is already in use has less risk.  Knowing that still does not tell a hacker that it is an account that is capable of signing in.  You can also take steps such as requiring a captcha or answering a math problem to ensure the registration form is not automated, before telling the person if the name is already in use.

If user profiles can be scanned on your site, such as the case with a social network, you can still secure the sign in names of your users by not displaying that as part of the user profile or using that as a value in any urls.  For instance, maybe you require sign in by email address, but only display their real name or a nick name.  Then only expose that sign in email address to people approved by that user or others that they follow.

III. SQL Injection

All user input should expect that a hacker is going to attempt SQL injection at some point.  If any single input in your application allows SQL injection to occur, then any database table accessible by that connection string is exposed to the hacker.  There are a number of options to prevent the SQL injection vulnerability.  If you are writing SQL code, use parameterized queries.  If you use stored procedures, do not use any parameter values to build query text inside the stored procedure.  If you are using an ORM, take a look at the queries it builds at run time to make sure it doesn’t break those rules.  For more mitigation options check out SQL Injection on wikipedia.

Another protection is to store the user and related tables in a separate database from the rest of the application.  Or use the same database, but connect with a different database user for any query against the security related database schema.  You may even want a dedicated database user for accessing the user table, and another dedicated one for the salt table, and yet another dedicated one for the security audit tables.  Make sure database security is setup such that only that database user can access the user and related security tables.  This reduces the chance the a SQL injection vulnerability elsewhere in your site or another site using the same databases can result in a leak of the user and salt tables.

Cryptographic Security

As the web developer, always assume the database will be stolen at some point.  Even with the best technological protections, at some point social engineering may occur; or someone may steal a laptop with a backup of the database.  Hopefully the network and database are secure, but you shouldn’t depend on it.

All this means is that passwords should be practically unrecoverable with the full database, and with or without the web application binaries and configuration.  You can’t make it impossible to ever crack a given hashed password, but what you can do it make every attempt as expensive as possible to make it impractical within the time it takes to warn users and let them reset their password to your site and any site reusing the same password.  A few things you can do to secure the passwords are:

  1. Use a strong hashing algorithm
  2. Use a truly random long salt for each user
  3. Use multiple hash iterations (ex. 3000-6000) to lengthen the time to create a hash from 500 to 1000 milliseconds.  The longer the better for security, but it must be balanced with usability. Update Sept 2017: use 6000-8000 iterations.
  4. Use a variable number of hash iterations per user (+/- 1  to 5%).
  5. Generate a new salt and hash for each user occasionally with a different salt and number of iterations.  At sign in is a good time to do this.
  6. Add an additional salt based on something not in the database.  This helps when only the database was compromised and not the location where your additional salt is stored.  Make this a configuration value that can be different for test and production environments.  A compromised test environment from a stolen laptop should not expose the additional salt used in production.

You should also plan for processor performance improvements when adhering to #1, #3 and #5.  You should be able to change the number of base iterations used over time to keep up with the latest hardware performance, and without invalidating current user credentials.  Anytime you upgrade to better application/web servers adjust your hash iteration average accordingly.  You should also be able to switch to a better hashing algorithm when the one you start out with  turns out to have a vulnerability.  To keep up you can change the base number of iterations (#3) over time.  That means you need to store a modifier per user to know how to adjust the base iteration to hash their password the correct number of times.

Implementation

You can find a reference implementation on the candor-common Github project.  It may evolve occasionally, so be sure to subscribe to changes.

Database Schema

The database schema can be found in detail in the candor-common Github project.  The tables should all be in a separate schema called Security, so that you can manage database user roles by schema.  The salt for each user should be stored in a separate table.  Additionally, if possible, storing the salt table in another database server would increase the surface that needed to be attacked before anyone could attempt to determine the hashed passwords.

Table AuthenticationHistory: records each sign in attempt to ensure sign in is not attempted to frequently for any given user name.  This slows down the speed at which someone can use the application to guess passwords with a dictionary attack.

  • UserName: the sign in name entered.  It may or may not be a user that exists. This is whatever the user entered, whether it was a valid user name format or not.
  • IPAddress: The IP address that connected to the application.  This could be spoofed by a hacker, but still worth logging.
  • IsAuthenticated: Stores if the session was authenticated or not.
  • SessionID: If the session was authenticated, this stores the related UserSession table record id.
  • CreatedDate: The date and time of the attempt (in UTC preferrably)

Table User: details about each user whether it is an active account or not.  This table can be seeded with user names you don’t want real users to have, if desired.  This should not be extended with user profile details.  Create a separate table in another schema for things like a first and last name or other details.

  • UserID: The unique record id for the user.  Internal use only.  Not meant to be displayed to users.
  • Name: The name used to sign in with, not a name for display publicly.  This may actually be an email address if you prefer.
  • PasswordHash: The final hashed value of the user’s password.  Never a raw password!
  • PasswordHashDate: The date and time when the password was hashed.  The application can use this to determine if the password should be hashed again at next sign in.
  • IsDeleted: specifies if this user can sign in or not.  Existence of a record prevents another user from using the same name.
  • plus audit columns

Table UserSalt: Details about a user’s salt and any hash algorithm options.  This is in a supplemental table to allow for it to  be moved to a secure database server separate from the user table.

  • UserID: The unique record id for the user.  Internal use only.  Not meant to be displayed to users.
  • Salt: The value used to salt the users password on each iteration.  This should be generated with a true random hash with a long value.
  • HashGroup: This determines how many times the password is hashed.  The simple implementation just adds this value to the base number of iterations the application code decides to use.  Without knowing that base number, this value does not help a hacker know how many times the password is actually hashed.
  • HashName: The code name of your choosing for which hashing provider is used to generate the hash for this user.
  • plus audit columns

Table UserSession: All the current authenticated sessions for each user, which may span weeks, one per device the user signs in from.  Historical user session’s get deleted on the next successful sign in or after a period of time.  This is not meant to be a permanent historical record of all sessions.

  • SessionID: The table unique identity.
  • UserID: The unique record id for the user.  Internal use only.  Not meant to be displayed to users.
  • RenewalToken: A GUID identifier stored in the user’s browser cookie collection for web, or other client device local storage for other application types.  This token should be passed to the application on every request to the server for validation and renewal.
  • ExpirationDate:  The date at which this token will expire or has expired.  This date is pushed into the future on each renewal, to be one of two durations from the current date and time.  One duration is for public access devices and the longer duration is for personally owned private devices.  The duration of each type is configured and controlled by the server application.
  • RenewedDate:  The date and time the session was last renewed.  The difference between this date and the expiration date will give a clue as to the session duration.
  • CreatedDate: the date when this session was first signed in with the password.  The application may have an optional rule that limits the maximum time a session can last from creation.

HashProvider

HashManager and HashProvider are abstractions to generating a hash.  This keeps the calling code from having to know how to generate a hash, encapsulating all hashing code into one location.  The current code comes with a SHA2HashProvider that uses System.Security.Cryptography.RNGCryptoServiceProvider to generate salts, and System.Security.Cryptography.SHA256Managed to generate hashes.  These are only supported on Windows Server 2008 and above and Windows Vista and above.  As better algorithms are created, new providers will be created.  The old providers should not be removed or changed as that would invalidate any previous hash generated by them.  Obsolete providers will be marked as such so that callers of the API have time to migrate to the newer providers.



Generally, you should access a hash provider by the provider Name, which is stored in the UserSalt table, HashName column.  The name of each provider ideally should be called something obscure and not the algorithm name so that someone viewing the HashName column value would also need the application configuration to know what algorithm the hash name refers to. Once you have the relevant HashProvider, you can call the Hash and GetSalt methods as needed during authentication, registration, and change user name or password.

Here is an example during authentication when you want to hash a raw password for verification against the stored hash.

HashProvider hasher = !string.IsNullOrEmpty(salt.HashName) ?
    HashManager.Providers[salt.HashName] : HashManager.Provider;
var passwordHash = hasher.Hash(salt.PasswordSalt, password,
    salt.HashGroup + BaseHashIterations);
if (user.PasswordHash != passwordHash)
    return FailAuthenticateUser(name, ipAddress, result);

Here is an example during registration when you want to generate a new salt and hash during setup of the initial user record.  When you call SelectProvider it selects a random, non obsolete, hash provider from the configuration.  Using the Random class is generally not best for a true random value, but it is ok to use here because we just need a relative even distribution of values between the minimum and maximum hash groups over time for all users.  The true random value in play here is the generated salt, which as of the current implementation uses RNGCryptoServiceProvider.

HashProvider hasher = HashManager.SelectProvider();
salt.PasswordSalt = hasher.GetSalt();
salt.HashGroup = new Random(DateTime.Now.Second).Next(HashGroupMinimum, HasGroupMaximum);
salt.HashName = hasher.Name;
user.PasswordHash = hasher.Hash(salt.PasswordSalt, password, salt.HashGroup + BaseHashIterations);
user.PasswordHashUpdatedDate = DateTime.UtcNow;

Application Configuration Options

BaseHashIterations –  This sets the base number of iterations that a password should be hashed.  The HashGroup on the UserSalt table record determines how many iterations plus or minus from that number should be used for that user.  This constant value is defined in the code, initially set at 5000.  This is expected to change over time, so it may be converted to a configuration option in the future.  When that happens it will default to the current hard-coded value.

HashGroupMinimum / HasGroupMaximum – These numbers define a range of values that should be used for a user’s HashGroup when generating hashes for new passwords.  This is only used during registration, when the user updates their password, and then only on authentication after success and it is determined that the user’s hash is out of date.

FailurePeriodMinutes – This is the period of time monitored for sign in failures for a given user name.

AllowedFailuresPerPeriod – The number of times a sign failure for a given user name is tolerated in the monitored period of time.  After that number every sign in is a failure until the failure period elapses.  There is a different failure message for this condition.

LoginExceededFailureMessage – The error message displayed when there have been too many sign in failures for a user name in the monitored period of time.  Even correct passwords result in this error message after failures are excessive during the monitored period.

LoginCredentialsFailureMessage – The error message displayed when either the user name does not exist or the password is incorrect.  The password expression is not checked during sign in, so this is the same error if the password rules during sign in are not correct as well.

PasswordExpiration – The amount of time before a password is considered expired and a user must enter a new password.  This can be left as zero for no password expiration.

There are also configuration options with regular expressions for valid password, name, and email formats.

Authentication Code

You can see the full authentication code in the Sql provider of the github repository. The authenticate method below is the private method used internally as part of registration, change password, and authentication.  Each of those methods pass in options for ‘checkHistory’, and ‘allowUpdateHash’ to optionally enable those blocks of code.  The public authenticate method passes in true for both options.

        private cs.UserIdentity AuthenticateUser(string name, string password,
            UserSessionDurationType duration, string ipAddress, bool checkHistory,
            bool allowUpdateHash, ExecutionResults result)
        {

The check history optional block gets the sign in attempts for the user name within the monitored failure period. If there are too many of them during the monitored period, then sign in is a failure. This is before we even check the password as correct.

            if (checkHistory)
            {
                var recentFailures = GetRecentFailedUserNameAuthenticationCount(name);
                if (recentFailures > AllowedFailuresPerPeriod)
                    return FailAuthenticateUser(name, ipAddress, result);
            }

This next block loads the user and the corresponding salt records. If either fail to load then the sign in fails. Note these call the same method to fail authentication as the too many failures in the monitored period block. That method sets the failure message displayed to the user and returns an anonymous identity.

            User user = GetUserByName(name);
            if (user == null)
                return FailAuthenticateUser(name, ipAddress, result);
            UserSalt salt = GetUserSalt(user.UserID);
            if (salt == null)
                return FailAuthenticateUser(name, ipAddress, result);

This next block gets the hash provider originally used to hash the stored password. Then it generates a hash to the entered password and checks if it matches what is stored in the user record. If not sign in fails the same as if the user did not exist above.

            //this should get a named hashProvider used to originally hash the password...
            //  fallback to 'default' provider in legacy case when we didn't store the name.
            HashProvider hasher = !string.IsNullOrEmpty(salt.HashName) ? HashManager.Providers[salt.HashName] : HashManager.Provider;
            var passwordHash = hasher.Hash(salt.PasswordSalt, password, salt.HashGroup + BaseHashIterations);
            if (user.PasswordHash != passwordHash)
                return FailAuthenticateUser(name, ipAddress, result);

This next section creates the session record and a corresponding successful authentication record.

            var session = new UserSession
            {
                CreatedDate = DateTime.UtcNow,
                ExpirationDate = DateTime.UtcNow.AddMinutes(duration == UserSessionDurationType.PublicComputer ? PublicSessionDuration : ExtendedSessionDuration),
                UserID = user.UserID,
                RenewalToken = Guid.NewGuid()
            };
            var history = new AuthenticationHistory
            {
                IPAddress = ipAddress,
                CreatedDate = DateTime.UtcNow,
                IsAuthenticated = true,
                UserName = name,
                UserSession = session
            };

Next, if the option is enabled, the code checks to see if the previous hash is too old. If it is too old it gets a new hash provider and uses it to generate a new salt and hash; and we store the name of that hash provider on the user salt.

            using (var scope = new System.Transactions.TransactionScope())
            {
                if (allowUpdateHash && (hasher.IsObsolete || user.PasswordHashUpdatedDate < DateTime.UtcNow.AddMonths(-1)))
                {
                    //update hashes on regular basis, keeps the iterations in latest range for current users, and with a 'current' hash provider.
                    hasher = HashManager.SelectProvider();
                    salt.PasswordSalt = hasher.GetSalt();
                    salt.HashGroup = new Random(DateTime.Now.Second).Next(HashGroupMinimum, HasGroupMaximum);
                    salt.HashName = hasher.Name;
                    user.PasswordHash = hasher.Hash(salt.PasswordSalt, password, salt.HashGroup + BaseHashIterations);
                    user.PasswordHashUpdatedDate = DateTime.UtcNow;
                    //starts as a lightweight transaction
                    SaveUser(user);
                    //enlists in a full distributed transaction if users and salts have different connection strings
                    SaveUserSalt(salt);
                }
                //either continues distributed transaction if applicable,
                //  or creates a new lightweight transaction for these two commands
                SaveUserSession(session);
                InsertUserHistory(history);
            }

The entities were updated above in a transaction. If the user and salt are using the database connection string then it is a lightweight database transaction; otherwise it escalates to a distributed transaction. For that to work you need to have the feature installed on your server (or development machine).

Finally, the authenticated user identity is returned to the caller. The caller will put it into the current authentication context which I will talk about another time.

            return new cs.UserIdentity(history, this.Name);
        }

Registration Code

On registration the first thing checked is that the user interface properly validated the entered user name and password to meet minimum requirements. These validation methods populate any error messages targeted at the user into the ExecutionResults instance; returning false if the values are invalid.

        public override UserIdentity RegisterUser(User user, UserSessionDurationType duration, String ipAddress, ExecutionResults result)
        {
            string password = user.PasswordHash;
            if (!ValidateName(user.Name, result) || !ValidatePassword(password, result))
                return new cs.UserIdentity();

Then the user name is checked if it matches an existing user record to prevent two users with the same sign in name. If you use email addresses for sign in names, this is less likely to be a problem.

            var existing = GetUserByName(user.Name);
            if (existing != null)
            {   //seed user table with deleted users with names you don't want users to have
                result.AppendError("The name you specified cannot be used.");
                return new cs.UserIdentity();
            }
            if (user.UserID.Equals(Guid.Empty))
                user.UserID = Guid.NewGuid();

Then a hash provider is selected at random, a salt and hash are generated with it, and all of those values are stored on the user and salt record.

            HashProvider hasher = HashManager.SelectProvider();
            var salt = new UserSalt
            {
                PasswordSalt = hasher.GetSalt(),
                UserID = user.UserID,
                HashGroup = new Random(DateTime.Now.Second).Next(HashGroupMinimum, HasGroupMaximum),
                HashName = hasher.Name
            };
            user.PasswordHash = hasher.Hash(salt.PasswordSalt, password,
                                                   salt.HashGroup + BaseHashIterations);

Next the user and salt are saved in a transaction.

            using (var scope = new System.Transactions.TransactionScope())
            {
                //starts as a lightweight transaction
                SaveUser(user);
                //enlists in a full distributed transaction if users and salts have different connection strings
                SaveUserSalt(salt);
            }

Finally, the private authenticate method is called to generate a user identity for the newly registered user. The option to allow hash update is not set since the hash was just generated. The option to check authentication history is not necessary since the user account was just created.

            return AuthenticateUser(name: user.Name, password: password, duration: duration,
                                    ipAddress: ipAddress, checkHistory: false, allowUpdateHash: false, result: result);
        }

Usage of Candor.Security

If you want to use the basic implementation of Candor.Security you can get the latest reference libraries via NuGet.  Just search for Candor to see a list of packages described in the implementation.  Look for the ones with mlang as the author to get the ones mentioned in this article.

Be aware that if you upgrade to a new version of the library that you read the release notes.  It may take a manual conversion process to upgrade so that old hashed passwords do not become unrecoverable.  One of the goals during package updates will be to make any incompatible updates based on configuration options, but there may be a change for which that is not possible.

Resources

I. Topical Explanations

Salted Password Hashing – Doing it Right.
http://crackstation.net/hashing-security.htm

Social Engineering (Wikipedia)
http://en.wikipedia.org/wiki/Social_engineering_(security)

Key Stretching (Wikipedia)
http://en.wikipedia.org/wiki/Key_stretching

How to “store” passwords securely
http://blog.barthe.ph/2012/06/15/howto-store-passwords/

Stored procedures and ORMs won’t save you from SQL injection
http://www.troyhunt.com/2012/12/stored-procedures-and-orms-wont-save.html

II. Reference Implementations

A reference Implementation project discussed above in C#.Net

https://github.com/michael-lang/candor-common

Here is another implementation that only misses a couple points, #5, #6, & #7, but it is part of the .net framework.  You could probably modify it to implement those points.

 “Stronger password hashing in .NET with Microsoft’s universal providers”
http://www.troyhunt.com/2012/07/stronger-password-hashing-in-net-with.html

Part 2 of this article – using it from an MVC4 application
http://candordeveloper.com/2013/02/02/securing-user-passwords-with-candor-security/

About the Author

Michael Lang

Co-Founder and CTO of Watchdog Creative, business development, technology vision, and more since 2013, Developer, and Mentor since 1999. See the about page for more details.

4 Comments