Hello Hartwig,
check that the permission and ownership of directory /usr/spool/sql and subdirectories and filesare correct, on UNIX the owner and group should be sdb:sdba
You should also look for other logs in the installation directory of SWPM and in MaxDB work directory which could make clear why the registration of the database instance failed.
Regards,
Valerio