I'm excited to have found new way in Apache Camel to build specific feature without writing any single code, which has always been my way due to time constraint and flexibility (to make everything configurable and every exception handled properly). It was a long winding process with many many rounds of trial and error.
What I'm trying to achieve here is very simple, just
read file and SFTP to the other side and Backup the original Source File! If this would have been done in a Java/Groovy and registered as a Spring Bean, then it shall be very fast. However I chose to do it the Camel-way!
- In your Camel-Context.xml (Spring DSL), declare a Route.
- The <From> should be the File EIP. Remember to set a "Delay" for the file polling interval.
- Next, use the SFTP EIP for the <To>
Done. So simple! However...
I want to LOG the entire process, such as:
- What Are the Files Being Picked Up?
- Whether the SFTP is Successful or Failed?
- Was the SFTPed file moved to a Backup folder?
For my own knowledge, I was using 2 ways to log:
- Camel's Log EIP - To log Route process + Error to the Application-specific log.
- Camel's Log - To log Error to the General Error log.
**For Error, will be written to both logs.
Camel's Log EIP
<camel:log message="[MyTimer2] SFTP Begins ${file:name}" loggingLevel="DEBUG" logName="myTimer2Logger" />
**${file:name} here is a kind of Reserved Keyword inside Camel's. It's called the File Expression language and this specific one will output the name of the file picked up in the log.
Camel's Log
<camel:to uri="log:error-rollingFileAppender?level=ERROR&showCaughtException=true&showStackTrace=true" />
** First Parameter (in this case - error-rollingFileAppender) refers to the Appender setup in my Application's LogBack.xml
Question, where should I place my LOG Error Block? I should only LOG Error upon Exception right?
First, I tried the <DoTry> - <DoCatch> way, I put my LOG Error function within <DoCatch>. However, there is a flaw here! Upon SFTP Exception, the File:// will still regard the SFTP as successful and continued to move file to the Backup folder.
Finally I found the right way, use <OnException>! So just put the Log Error block inside your <OnException>, which is inside the <Route>!
<camel:exception>java.lang.Exception</camel:exception>
<camel:to uri="log:error-rollingFileAppender? level=ERROR&showCaughtException=true&showStackTrace=true" />
<camel:log message="[MyTimer2] File Transmission Error ${exception.stacktrace}" loggingLevel="ERROR" logName="myTimer2Logger" />
</camel:onException>
Is that all? Nope.
I wanted to Retry for a fixed number of times upon any kind of Exception!
Easy. Just declare the below:
<camel:redeliveryPolicyProfile id="myRedelivery" retryAttemptedLogLevel="ERROR" maximumRedeliveries="${H2MaxRetry}" redeliveryDelay="${H2RetryDelay}" />
Then add this as the "redeliveryPolicyRef" to your Route.
Last but not least, I wanted to set all the properties, such as the Input Directory, the SFTP host, username, password and etc in a single properties file!
Here's the trick:
Use <Endpoint> for your File:// and SFTP://
Somehow only Endpoint allows Spring Property Placeholder!
<camel:endpoint id="send_source" uri="file://${app.dir}/filewatcher/anz_output?delay=${myTimer2}&move=${app.dir}/filewatcher/anz_backup/$simple{file:name}.done&moveFailed=${app.dir}/filewatcher/anz_bad"/>
<camel:endpoint id="send_remote" uri="sftp://${H2hostname}${H2Path}?username=${H2user}&password=${H2password}&soTimeout=${H2SocketTimeout}&binary=${H2Binary}"/>
**These properties placeholder are stored in the properties file configured at the PlaceholderConfigurer
