Friday, January 29, 2016

RIDC: Checkin your file to Oracle HCM Cloud

In this blog I will talking about my findings of using Oracle HCM Cloud as storage. You can actually checkin/checkout your files here. There are various reasons you may want to store your files there. Few could be
1. FBL
2. HDL
3. You Paas Application needs to store some file (Although I will prefer storage cloud but then you might need a separate license. Why not to use UCM of Oracle HCM Cloud itself).

Before coming to actual implementation, I would like to clear few things.

1. If saas url is https://hcm-abc.oracleoutsourcing.com/homePage/faces/AtkHomePageWelcome, What will be content-server url
Ans: https://fs-abc.oracleoutsourcing.com/cs/idcplg (Same SaaS username/password will work)


2. As we are going to use RIDC apis we need to understand supported protocols:

Intradoc (with SSL also): This is a direct connection between client and server. Server must be configured to accept request from selected clients. You must be setting IntradocServerPort and SocketHostAddressSecurityFilter in WCC_domain/ucm/cs/config/config.cfg file or (Web UI hos:port/cs/idplg >> Administration >> Admin Server >> General Configuration




As this connection is done over trusted network, it does not need any password authentication. Only provide username and do transaction.

IdcContext userContext = new IdcContext("HCM_ADMIN"); //No need of password.
Intradoc communication can also be done over SSL. You should do below line of coding for that.
// create the manager
IdcClientManager manager = new IdcClientManager();
// build a client that will communicate using the intradoc protocol
IdcClient idcClient = manager.createClient("idc://localhost:4444");
// set the SSL socket options
IntradocClientConfig config = (IntradocClientConfig) idcClient.getConfig();
config.setKeystoreFile("ketstore/client_keystore");  //location of keystore file
config.setKeystorePassword ("password");      // keystore password
config.setKeystoreAlias("SecureClient");  //keystore alias
config.setKeystoreAliasPassword("password");  //password for keystore alias

In case of SaaS, intradoc may not be useful as we can’t configure SocketHostAddressSecurityFilter property of UCM server.

http/https: RIDC communicates with the web server for Oracle Content Server using the Apache HttpClient package. Unlike Intradoc, this protocol requires authentication credentials for each request. It does not require any configuration on server side.
If you want to use http/https, you need to use url as 
// create the manager
IdcClientManager manager = new IdcClientManager();

IdcClient idcClient = manager.createClient("https://host.oracleoutsourcing.com/_dav/cs/idcplg");
//NOTE: we have used _dav in url. Otherwise you will receive //oracle.stellent.ridc.protocol.http.HttpProtocolException: Http status: //HTTP/1.1 302 Moved Temporarily //ERROR

//If you are using https, you need to set truststores as 
System.setProperty("javax.net.ssl.trustStore", "D:/jks/Cert_DER.jks");
System.setProperty("javax.net.ssl.trustStorePassword","mypassword");

//For usercontext you should set AuthScheme as Basic
IdcContext userContext = new IdcContext("username", "password");             
userContext.setAuthScheme(IdcContext.HttpAuthScheme.BasicAuth);

JAX-WS: The JAX-WS protocol is supported only in Oracle WebCenter Content 11g with Oracle Content Server running in Oracle WebLogic Server. To provide JAX-WS support, several additional JAR files are required.


NOW COMING TO THE MAIN PART: Steps to import a file on Oracle HCM Cloud (UCM) server

1. No special configuration is needed on server side
2. Get url to make connection. It should be https://host.oracleoutsourcing.com/_dav/cs/idcplg
Replace host with your environment and don’t forget _dav
3. Get username/password who has access to UCM. You can use your SaaS username/password to login to https://host.oracleoutsourcing.com/cs
If you are able to login successfully then check if you can upload/download documents etc. If you can do it through UI, you should be able to do it programmatically.
4. Create keystore and install certificate. (THIS STEP IS IMPORTANT and ERROR PRON)
  • Launch https://host.oracleoutsourcing.com/cs in IE
  • Install certificate in browser 
    Lock-Icon-in-url >> View Certificate >> Install Certificate
            

  • Export certificate from browser Tools >> Internet Options >> Content >> Certificate >> Other People >> Select Certificate >> Export

  • Import downloaded certificate in a keystore
  • If you do not have a keystore already and you want to create new keystore altogether.  
    NOTE: In case of weblogic web program you may want to get existing keystore and then add new certificate instead of creating new keystore. 
    keytool -import -file D:\jks\Cert_DER.cer -alias orakey -keypass mypassword -keystore D:\jks\Cert_DER.jks -storepass mypassword
    This command will cerate a keystore (jks file) with a key (named orakey). Password for key and keystore is same as ‘mypassword’. Certificate is imported in this keystore. 

    If you already have keystore and you want to import certificate in it 
    keytool -importcert -file D:\jks\Cert_DER.cer –keystore D:\oldjks\Cert_DER.jks -alias orakey -storepass <whatever-is-keystore-password>

    If you want to list keystore data you can 
    keytool -list -v -keystore D:\jks\Cert_DER.jks -storepass mypassword

Following above keystore steps your keystore should be ready with appropriate certificate.

Now we need to write a java code with RIDC apis and configure it to use jks file we just created. 
NOTE: If you are using weblogic server then you should configure weblogic keystore instead of referring jks from code.

5. Java Code: Let say you have placed your jks as D:/jks/Cert_DER.jks and file that you want to import at D:/file/test.txt

import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.IOException;
import oracle.stellent.ridc.IdcClient;
import oracle.stellent.ridc.IdcClientException;
import oracle.stellent.ridc.IdcClientManager;
import oracle.stellent.ridc.IdcContext;
import oracle.stellent.ridc.model.DataBinder;
import oracle.stellent.ridc.model.TransferFile;
import oracle.stellent.ridc.protocol.ServiceResponse;

public class RIDCUtil {
      public static void main(String[] arg) throws Exception {
             try {
                            IdcClientManager m_clientManager = new IdcClientManager();
                            IdcClient idcClient = m_clientManager.createClient("https://host.oracleoutsourcing.com/_dav/cs/idcplg");
                            IdcContext userContext = new IdcContext("username", "pwd");
 userContext.setAuthScheme(IdcContext.HttpAuthScheme.BasicAuth);
 System.setProperty("javax.net.ssl.trustStore", "D:/jks/Cert_DER.jks");
           System.setProperty("javax.net.ssl.trustStorePassword","mypassword");
 checkin(idcClient, userContext, "D:/file/test.txt", // Replace with fully qualified path to source file
          "Document", // content type
          "Test Document to check RIDC", // doc title
           userContext.getUser(), // author
           "FAFusionImportExport", // security group
           "hcm$/dataloader$/import$", // account
           "TEST_DOC_FOR_RIDC") // dDocName - this is the ContentId
                        ;
        } catch (IdcClientException e) {
                e.printStackTrace();
            }
    }

/** * Method description * *
     * @param idcClient *
     * @param userContext *
     * @param sourceFileFQP fully qualified path to source content *
     * @param contentType content type *
     * @param dDocTitle doc title *
     * @param dDocAuthor author *
     * @param dSecurityGroup security group *
     * @param dDocAccount account *
     * @param dDocName dDocName * *
     * @throws IdcClientException */
    public static void checkin(IdcClient idcClient, IdcContext userContext,
                               String sourceFileFQP, String contentType,
                               String dDocTitle, String dDocAuthor,
                               String dSecurityGroup, String dDocAccount,
                               String dDocName) throws IdcClientException {
                                   
        InputStream is = null;
        try {
            String fileName = sourceFileFQP.substring(sourceFileFQP.lastIndexOf('/') + 1);
            is = new FileInputStream(sourceFileFQP);
            long fileLength = new File(sourceFileFQP).length();
            TransferFile primaryFile = new TransferFile();
            primaryFile.setInputStream(is);
            primaryFile.setContentType(contentType);
            primaryFile.setFileName(fileName);
            primaryFile.setContentLength(fileLength); // note!!! when using HTTP protocol (not intradoc/jaxws) - one must explicitly // set the Content Length when supplying an InputStream to the transfer file // e.g.
            primaryFile.setContentLength(fileLength); // otherwise, a 0-byte file results on the server
            DataBinder request = idcClient.createBinder();
            request.putLocal("IdcService", "CHECKIN_UNIVERSAL");
            request.addFile("primaryFile", primaryFile);
            request.putLocal("dDocTitle", dDocTitle);
            request.putLocal("dDocAuthor", dDocAuthor);
            request.putLocal("dDocType", contentType);
            request.putLocal("dSecurityGroup", dSecurityGroup);
            // if server is setup to use accounts - an account MUST be specified
            // // even if it is the empty string; supplying null results in Content server
            // // attempting to apply an account named "null" to the content!
            request.putLocal("dDocAccount", dDocAccount == null ? "" : dDocAccount);
            if (dDocName != null && dDocName.trim().length() > 0) {
                request.putLocal("dDocName", dDocName);
            }
            // execute the request
            ServiceResponse response = idcClient.sendRequest(userContext, request); // throws IdcClientException
            // get the binder - get a binder closes the response automatically
            DataBinder responseBinder = response.getResponseAsBinder(); // throws IdcClientException
        } catch (IOException e) {
            e.printStackTrace(System.out);
        } finally {
            if (is != null) {
                try { is.close();
            } catch (IOException ignore) {
                    }
                }
            }
        }
    }

If you are making your RIDC call using a web program deployed on weblogic server, you should not set javax.net.ssl.trustStore  and javax.net.ssl.trustStorePassword property in java code but you should configure jks file in weblogic.



6. Above code can be improved in many way. You may want to checkin file in a particular folder, you may want to see if file already present, You may want to handle exception gracefully, you may want to take all hardcoded strings (ucm-url, ucm-username/password, keystore-location/password, content-type, security-group, account etc.) to a property file. Here it is just a POC.


7.  To run java program from command prompt, generate a jar file containing RIDCUtil (say MyIntegrationJar.jar). Place all required jars in one directory and run below command from that directory.

C:\jdk1.6.0_31_64\bin\java -classpath .\MyIntegrationJar.jar;.\oracle.ucm.ridc-11.1.1.jar;.\commons-httpclient-3.1.jar;.\commons-logging-1.0.4.jar;.\commons-codec-1.2.jar RIDCUtil

8. You can verify document in UCM by login into UCM https://host.oracleoutsourcing.com/cs and searching the document 



Below are the various issue that you may face while implementing it. Most of them are actually certificate related


Issues Solutions
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:183)
at java.security.cert.PKIXParameters.(PKIXParameters.java:103)
at java.security.cert.PKIXBuilderParameters.(PKIXBuilderParameters.java:87)
at sun.security.validator.PKIXValidator.(PKIXValidator.java:55)
jks file cannot be located. Check if you have given correct path for jks file.NOTE: On windows you should use / instead of \\.
Caused by: java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: com.sun.net.ssl.internal.ssl.DefaultSSLContextImpl)
at javax.net.ssl.DefaultSSLSocketFactory.throwException(SSLSocketFactory.java:179)
at javax.net.ssl.DefaultSSLSocketFactory.createSocket(SSLSocketFactory.java:212)

Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: com.sun.net.ssl.internal.ssl.DefaultSSLContextImpl)
at java.security.Provider$Service.newInstance(Provider.java:1245)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:220)

Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:771)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:38)
at java.security.KeyStore.load(KeyStore.java:1185)

Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:769)
... 30 more
keystore or key password is wrong. I generally keep keystore and key password same and provide that.
oracle.stellent.ridc.protocol.http.HttpProtocolException: Http status: HTTP/1.1 302 Moved Temporarily
at oracle.stellent.ridc.protocol.http.IdcHttpProtocol.writeRequest(IdcHttpProtocol.java:407)
at oracle.stellent.ridc.IdcClient.sendRequest(IdcClient.java:179)
Use _dav in url. By default cs/idcplg url is for web access and it redirects user to login page. If you add _dav in url, then you can pass username/password with request and they will be used to login.
oracle.stellent.ridc.protocol.ProtocolException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at oracle.stellent.ridc.protocol.http.IdcHttpProtocol.sendRequest(IdcHttpProtocol.java:661)
at oracle.stellent.ridc.protocol.http.IdcHttpProtocol.sendRequest(IdcHttpProtocol.java:623)
at oracle.stellent.ridc.protocol.http.auth.BasicAuthHandler.sendAuthenticatedRequest(BasicAuthHandler.java:91)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
…..
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:323)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:217)
at sun.security.validator.Validator.validate(Validator.java:218)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:126)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:209)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:249)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1185)
... 31 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:318)
... 37 more
It means, Java has located keystore but using that keystore its not able to make handshake. There could be multiple reasons for that
  1. If you are using standalone java program, you might not have added below java code System.setProperty("javax.net.ssl.trustStore", "D:/jks/Cert_DER.jks"); System.setProperty("javax.net.ssl.trustStorePassword","mypassword");
  2. If you are using web program on weblogic, You might not have configured your jks as keystore.
  3. Your path of jks is wrong
  4. For windows also you should use / to define path. You might have used \\ or \
  5. Your jks file does not have certificate that is needed to communicate with server. Verify it using keytool –list command
  6. Your jks file has expired certificate. Verify it using keytool –list (Valid From and Valit To). If expired, download new cer, import it in keystore and mention new keystore in code.
oracle.stellent.ridc.protocol.ServiceException: Content item 'TEST_DOC_FOR_RIDC' was not successfully checked in. The content item is not currently checked out.
at oracle.stellent.ridc.protocol.ServiceResponse.getResponseAsBinder(ServiceResponse.java:142)
at oracle.stellent.ridc.protocol.ServiceResponse.getResponseAsBinder(ServiceResponse.java:108)
It means content is already present in server and it must be checkedout before check in. Ideally you should search content first using GET_SEARCH_RESULTS service and dIsCheckedOut metadata. If content is not checked out you should check-out first and then check-in.
org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
INFO: basic authentication scheme selected
org.apache.commons.httpclient.HttpMethodDirector processWWWAuthChallenge
INFO: No credentials available for BASIC 'UCM WebDAV Server'@host.oracleoutsourcing.com:443
These are just warnings but its because you missed to set authentication scheme userContext.setAuthScheme(IdcContext.HttpAuthScheme.BasicAuth);

Tuesday, January 26, 2016

MAF: Invoking WebServices (Which way to choose)

In MAF there are actually multiple ways to create UI based on webservice.

Your choices are
1. If service is SOAP based, you can create a datacontrol and use it directly to build UI
2. If service is SOAP based, you can create a datacontrol, Call datacontrol from POJO using ADFmfJavaUtilities.invokeDataControlMethod, Expose POJO as datacontrol and use it to build UI.
3. If service is REST based and output in XML, you can create a datacontrol and use it directly to build UI.
4. If service is REST based and output is XML, you can create a datacontrol, call datacontrol from POJO using ADFmfJavaUtilities.invokeDataControlMethod, expose POJO as datacontrol and use it to build UI.
5. If service is REST based and output is JSON, you can create a POJO, call webservice directly using RESTServiceAdapter class. Expose POJO as datacontrol and use it to build UI.

Below diagram depicts above mentioned choices a flow

Although we have multiple choices to invoke webservices, but I only go with two approaches. It reduces my confusion and provide me best options.

1. I do not drag and drop webservice datacontrol on UI. It creates few problems
     a. I lose ability to pre-process/validate/post-process/cache-result capabilities. These capabilities comes only if we invoke webservice datacontrol using POJO. So my preferred way is always have a POJO wrapper around service.

   b. If you directly drag and drop webservice datacontrol on UI, it may get executed more frequently than you want. At times you need to provide logic on UI rendered property or binding iterator to stop auto execution of webservice. It complicates thing.

  So my first suggestion do not drag and drop webservice datacontrol directly on UI, instead wrap it around a POJO and expose POJO as datacontrol. Inside POJO you can do multiple things and invoke service using AdfmfJavaUtilities.InvokeDataControlMethod if needed.

2. For REST service I do not go datacontrol way at all. I prefer simple POJO with RESTServiceAdapter class.
   a. If REST is XML based I have ability to create a datacontrol and then invoke it using AdfmfJavaUtilities but I do not prefer this approach. Its like completely defining your REST service at client side, what is possible url, what are output their attributes etc. If anything changes, I need to recreate datacontrol and it may be lots of testing again. But if I use RESTServiceAdapter, its either no impact or minimal impact on my code.


Effectively my way of invoking services in MAF is pretty straigt forward
1. If service is SOAP
       a. Create SOAP datacontrol
       b. Create POJO and use AdfmfJavaUtilities class to invoke webservice. Also implement pre-processing/validation-of-input/post-processing/caching-of-result logic in POJO.
       c. Expose POJO as datacontorl
       d. Create UI using POJO datacontrol

2. If service is REST
     a. Create POJO (No need to create datacontrol using REST service)
     b. Use RESTServiceAdapter class to invoke rest service from POJO. Also implement pre-processing/validation-of-input/post-processing/caching-of-result logic in POJO.
     c. Expose POJO as datacontrol
     d. Create UI using POJO datacontrol


Above two approaches are my preferred way.

Thursday, January 21, 2016

JSON Mesasge Structure

In this blog I am going to write about JSON message. A json message is very similar to a JavaScript objects/arrays. Below are few rules I can define for a JSON message

1.       Message should start with a curly braces to represent a JSON object or square bracket to represent JSON Array.

Simplified JSON object is
{“key1”: value, “key2”: value} //you can add any number of key-value pair

Simplified JSON array could be
[value1, value2, value3] // you can add any number of values

2.       As per point 1, we have keys and values in a JSON message.

3.       Values can be null, primitive (string, Boolean, number), object or collection.

4.       Null value should be clearly represented by keyword null. No need of any double quotes.

5.       Any String value in message must be within double quotes

6.       Any Boolean value must be without double quote. Only two value expected true/false

7.       Any number value must be without double quote. It must be a number.

8.       Any object value must be surrounded by curly brackets { “key” : value, “key2”: value}. Object should have key/value pair. Key is a string so its name should be in double quotes. Key value must be separated by colon (:). Value could be null, primitive (string, Boolean, number), object {…} or collection [value, value, value]. Comma should be used to separate two key/value combination.

9.       Any collection value must be surrounded by square bracket […]. It can only have values in it. Effectively structure would be like [value1, value2, value3]. Values again could be null, primitive, object or collection.


Sample JSON object
    { “name”: “Development”, “dept-id”: 100,
“employees”: [
                {“first-name”: “Sanjeev”, “middle-name”: null, “empid”: 100, “isActiveEmployee”: true},
{“first-name”: “Sachin”, “middle-name”: null, “empid”: 101, “isActiveEmployee”: false},
         ]
    }

Sample JSON Array object:
[
                {“first-name”: “Sanjeev”, “middle-name”: null, “empid”: 100, “isActiveEmployee”: true},
{“first-name”: “Sachin”, “middle-name”: null, “empid”: 101, “isActiveEmployee”: false},

         ]