A recent approach yielded the question, how do I do public key authentication between this AIX host running OpenSSH and a SCO host running Tectia SSH2?

The answer lies in converting the public key you generated on the source server (AIX) into a format that SSH2 on the target servers (SCO) can understand.

If, for example, you had generated a DSA key on AIX called id_dsa, it will also have generated a corresponding public key called id_dsa.pub. It is this key we want to convert for use on SCO. We can do this using the command:

ssh-keygen -e -f id_dsa.pub >id_dsa_ssh2.pub

We now have a new file called id_dsa_ssh2.pub that needs to be put in the SCO hosts .ssh2 directory, and a reference to it amended to the SCO hosts .ssh2/authorization file. For example: –

Key id_dsa_ssh2.pub

The usual rules regarding the identification and authorization files apply to the SCO host. Do not rename the id_dsa key on the AIX host, as (I believe) it will only accept the filenames identity (identity.pub), id_rsa (id_rsa.pub) and id_dsa (id_dsa.pub).

UPDATE : If you want to do this the other way ’round, i.e. use public key authentication from SSH2 to OpenSSH you need to convert the SSH2 public key generated for use on the OpenSSH server. Do this with:

ssh-keygen -i -f ssh2_publickey ~/.ssh/authorized_keys2

Obviously you want to create the ssh2_publickey file containing the public key of the server requesting access.