Export password hashes in the PHC format

documentation sysadmin
,
1

Current details about Discourse’s password storage system can be found in discourse/SECURITY.md at main · discourse/discourse · GitHub. At the time of writing, we use PDKDF2-SHA256, with 64k iterations.

In some situations, you may want to export all of the hashed passwords from Discourse, and import them into another system. For example, you may be migrating from built-in Discourse authentication to a custom SSO system. Remember, it is impossible to extract the original passwords from the database, so your destination system needs to be capable of running the same hashing algorithm with the same parameters.

You can use data explorer to export the information in a computer-readable format:

SELECT id, username, salt, password_hash FROM users

This will export the data in the native Discourse format. The salt and password hash will be hex encoded.

Some external systems support the PHC string format which aims to be a cross-algorithm way to represent the output of a password hashing function. For pbkdf2-sha256, this string contains the algorithm type, the number of iterations, base64-encoded salt and base64-encoded hash. Fortunately, Postgres can handle all of this for us in a single query.

To generate PHC strings for each Discourse user, you can use a data explorer query like this:

SELECT id, username,
  concat(
    '$pbkdf2-sha256$i=64000,l=32$',
    salt,
    '$',
    replace(encode(decode(password_hash, 'hex'), 'base64'), '=', '')
  ) as phc
FROM users

If you use Auth0 then you want this instead:

SELECT
    user_emails.email,
    users.active as email_verified,
    concat(
        '$pbkdf2-sha256$i=64000,l=32$',
        replace(encode(users.salt::bytea, 'base64'), '=', ''),
        '$',
        replace(encode(decode(users.password_hash, 'hex'), 'base64'), '=', '')
    ) as password_hash
FROM users
INNER JOIN user_emails 
ON users.id = user_emails.user_id 
AND user_emails.primary IS TRUE
AND users.id > 0
13 Likes
2

Just a note here that I (with some help from the friendly Auth0 team) ended up tweaking the example query to generate valid PHC strings for importing user passwords into Auth0.

I also encoded the salt as base64 by changing this line

salt,

to

replace(encode(users.salt::bytea, 'base64'), '=', ''),

See further here (including a step by step on how to import Discourse users and their passwords into Auth0).

1 Like
3

Thanks @angus - this is interesting because we have had a customer use the query in the OP to successfully import users to Auth0. I wonder if something has changed in their import process - IIRC the ability to import PHC strings to Auth0 was very new back in November :thinking:

3 Likes
4

Yeah, I was wondering about that, and thought the same.

I also wasn’t quite sure of the language in the PHC specification. Not sure if this means the salt must be B64 encoded or not.

The salt consists in a sequence of characters in: [a-zA-Z0-9/+.-] (lowercase letters, uppercase letters, digits, /, +, . and -). The function specification MUST define the set of valid salt values and a maximum length for this field. Functions that work over arbitrary binary salts SHOULD define that field to be the B64 encoding for a binary value whose length falls in a defined range or set of ranges.

2 Likes
5

Sorry for getting off-topic, but has anyone imported password hashes from Auth0 to Discourse? I’m thinking of doing this migration so any help would be appreciated. I’m not a paying Auth0 customer so I just wanted to know if this is feasible before paying for the password hash export.

Thanks.

6

Importing passwords is not supported by Discourse core, although it might be possible using an adapted version of this third-party plugin:

4 Likes
7

Thank you! I’ve found the repo on GitHub but not the topic here on meta.

1 Like
9

Hey @angus, we’re cleaning things up here.

Is it true that the OP code block should read:

SELECT id, username,
  concat(
    '$pbkdf2-sha256$i=64000,l=32$',
    replace(encode(users.salt::bytea, 'base64'), '=', ''),
    '$',
    replace(encode(decode(password_hash, 'hex'), 'base64'), '=', '')
  ) as phc
FROM users

and include something like:

For more information about importing Discourse passwords to Auth0 see Bulk User Import Custom Password Hash Issue - Auth0 Community.

To move data from Auth0 to Discourse, this might help: Migrated password hashes support.

3 Likes
10

Yup, that looks good.

You may want to add something in about each import needing specific attention, as the user data being handled will differ depending on the use case, i.e. don’t just copy / paste these queries.

Also, exporting passwords in PHC is not necessarily only for Auth0, so perhaps that should just be refered to as an “example”.

The full query I used for my export was

SELECT
    user_emails.email,
    users.active as email_verified,
    concat(
        '$pbkdf2-sha256$i=64000,l=32$',
        replace(encode(users.salt::bytea, 'base64'), '=', ''),
        '$',
        replace(encode(decode(users.password_hash, 'hex'), 'base64'), '=', '')
    ) as password_hash
FROM users
INNER JOIN user_emails 
ON users.id = user_emails.user_id 
AND user_emails.primary IS TRUE
AND users.id > 0
2 Likes