Talend Password Storage routines
pwdStore password storage
In ETL jobs it is extremely common to have one or more passwords to set in order to connect to a db server, an ftp server, a webservice etc.
These passwords in TOS are normally passed to specific paramenters of the components.
Unfortunately values entered in the component's parameters are converted into constants during the compilation that happens just before the job execution and these constants are placed in the generated java code and eventually readable from there.
Worse, if they are embedded in the repository metadata, they are stored in plain text in xml files.
An additional issue is realted to the need to deploy jobs in different environments (i.e. dev/test/prod or for different customers etc), this requires to re-set all the connection details each time the environment is changed.
An easy solution to this is to use an external configuration file and load the needed values in the context, however also this solution is a security risk because all the passwords would be stored in a plain text file.
Java provides strong crypting capabilities that can be used to store passwords (or other strings) in an encrypted external file.
A set of alias/password pairs can be encrypted using java.crypto algorithms and then stored in a file on disk.
Tho read and write data to this file an encryption / decryption password is needed and this would be the only password to be passed to the talend job.
Indeed someone in possess of the crypted file, the password and the routines could manage to read the password list, however the encrypted password could be stored in a safe location and replaced when one or more passwords must be updated.
The advantage is that no connection passwords would be visible in the java code itself, nor in the TOS job or repository files, moreover if the job designer can retrieve dynamically and with a secure channel the single encryption password, then the system is fully secure.
A simple routine class (using an encryption library, deployed in a jar file) is provided to ease the tasks of reading and writing password pairs.
Version : 0.1
Release Date : June 2011
Status : Beta
The job pwdTest demomstrates the usage of the password storage routines
The preJob chain is normally used to open the database connections, since here a password is normally provided as a parameter, we substitute it with a password dynamically retrieved from an encrypted file, where it was previously stored.
The tJava component set PwdStore contains a single line of code that instructs the pwdStore routines to use a specific password file and an encryption key.
Once the password Storage parameters are set, passwords can be retrieved using the method pwdStore.getPwd(alias) where the "alias" is a convenient name we used to identify a specific password, i.e. for the root user password of the local mysql database it could be : mysql.local.root.pwd
In this example the password is dynamically retrieved from within the tMysqlConnection_1 password parameter.
The routines provide also a method to write/update password pairs :
pwdStore.setPwd(String fname,String kspassword,String alias, String pwd);
pwdStore.setPwd(String alias, String pwd)
The first one does not require to init the pwdStore, while the second requires the same kind of approach used in the example (with the pwdStore.setPwdStore line in the preJob chain).
To ease the management of the password file, a simple java executable class setpwd is provided to add, remove update or read the passwords, it's an extremely basic tool, so don't expect much from it.
You can run it having the included jar in your classpath, here you can find few usage exaples ;
Add or modify a password (if the file does not exist it is created, else the new pair will be appended).
java setpwd -a /home/franz/test.pwd 1234567890 mysql.local.root.pwd mysecretpassword
Remove a pair
java setpwd -d /home/franz/test.pwd 1234567890 mysql.local.root.pwd
Read a password
java setpwd -r /home/franz/test.pwd 1234567890 mysql.local.root.pwd
List (decrypted) pairs in the file
java setpwd -l /home/franz/test.pwd 1234567890
The current version of the tool does not allow to use spaces in any parameter, moreover it is not possible to use the char "=" in an alias
THIS SOFTWARE IS PROVIDED BY POWERUP ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL POWERUP BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.