"traditional private key format" vs PKCS8 - c

Question

The man page for PEM_write_PrivateKey states that it writes the private key using the traditional private key format.
How is this related to PKCS8 and PKCS1?

Answer 1

The documentation that you are referring to does not seem to be accurate (anymore). Nowadays, PEM_write_PrivateKey() does the same thing as PEM_write_PKCS8PrivateKey() for the OpenSSL implementations of RSA, DSA and EC keys.
As often the case with OpenSSL, the source code is a more reliable way to get your information. Looking at PEM_write_bio_PrivateKey(), which is supposed to give the "traditional" format according to that documentation, we see:
int PEM_write_bio_PrivateKey(BIO *bp, EVP_PKEY *x, const EVP_CIPHER *enc,
unsigned char *kstr, int klen,
pem_password_cb *cb, void *u)
{
if (x->ameth == NULL || x->ameth->priv_encode != NULL)
return PEM_write_bio_PKCS8PrivateKey(bp, x, enc,
(char *)kstr, klen, cb, u);
return PEM_write_bio_PrivateKey_traditional(bp, x, enc, kstr, klen, cb, u);
}
Indeed there is mention of a traditional private key write function, but only if the priv_encode method is not implemented for that particular key type. But it actually is implemented for the standard key types. For 1.0.2g, the version you are using according to your comment below, you can see the associated functions here: rsa_priv_encode(), dsa_priv_encode() and eckey_priv_encode(). PKCS#8 is the standard format when writing private keys.
PKCS#8 is capable of capturing multiple kinds of keys. Its format includes the option to store private keys in an encrypted form. This is different from older formats, where the encryption of the key happened at the PEM level, using a weaker encryption schema. See this answer to the SO question Password callback for reading public key with OpenSSL API for a more elaborate explanation on that in the case of RSA, comparing it to PKCS#1.
The "traditional" key format in your question refers to non-PKCS#8 key formats, which are standard in the case of RSA and EC keys and OpenSSL-specific for DSA, but are not uniform. For RSA keys, that happens to be a format often referred to as the PKCS#1 format. See also this answer to the SO question PKCS#1 and PKCS#8 format for RSA private key for more information.
If you want to write in the "traditional" formats, you will have to explicitly invoke the associated functions, like for example PEM_write_RSAPrivateKey(). In this case, the documentation does seem accurate (except for the grammatical error :-) ):
The RSAPrivateKey functions process an RSA private key using an RSA
structure. The write routines uses traditional format.