Showing posts with label UNIX. Show all posts
Showing posts with label UNIX. Show all posts

Friday, January 5, 2018

Day 24 The tide is high

Day 24

The tide is high

Let us say that you do not have Sterling Control Center to run the High Water Mark report for maximum concurrent sessions.
You might want this for determining if you are making optimum use of the number of concurrent sessions that you are licensed for.
You may not have enough Connect:Direct nodes that you feel justifies the need for Sterling Control Center.

For Windows

The following script water-mark.vbs is an example of how to do this using VBScript and the Connect:Direct Windows SDK.
In fact this is a fairly simple example of how to use the Connect:Direct Windows SDK.
'
' Script to report on the high water mark for the number of concurrent Connect:Direct sessions 
'

Dim node        ' Represents the Connect Direct Node we are using. 
Dim stats       ' A collection of statistic records. 
Dim stat        ' An individual statistic record. 

' Make the OLE/COM connection to Connect:Direct 
Set node = CreateObject("CD.Node")

' Sign on using defaults from the Client Connection Utility
node.Connect "CD.NICKE","",""

' Get start and end sessions statistics records since yesterday
set stats = node.SelectStats("select statistics startt=(today) recids=(SSTR,SEND) ccode=(eq,0)")

currentNumberOfSessions = 0
highWaterMark  = 0

For Each stat in stats
       Select Case stat.RecId 
          Case "SSTR" currentNumberOfSessions=currentNumberOfSessions+1          Case "SEND" currentNumberOfSessions=currentNumberOfSessions-1       End Select
       If currentNumberOfSessions > highWaterMark Then
          highWaterMark = currentNumberOfSessions
       End If
Next

Wscript.echo "Concurrent sessions high water mark = " & highWaterMark

Set node  = nothing
Set stats = nothing
Set stat  = nothing
If you have the Connect:Direct Windows SDK installed and default sign on credentials registered with the Client Connection Utility you can run the script as follows:
C:\Users\nicke> cscript /nologo water-mark.vbs

Concurrent sessions high water mark = 5
The above script is minimal in that it is only meant to be run interactively as it does not itself check for every error.
You run the script on a Windows machine with the SDK installed, but the node in question can be a remote Windows or UNIX (I have not tested other platforms) Connect:Direct node as long as the node is registered with the Client Connection Utility.

For UNIX

If all you have are UNIX machines then you can use the following shell function for convenience:
function watermark
{
        # Usage: cd work/UNIX.NODE ; cat S201712* | watermark
        egrep "RECI=(SSTR|SEND)" | grep CCOD=0 | awk '
BEGIN{
        currentNumberOfSessions=0
        highWaterMark=0
}
/RECI=SSTR/ { currentNumberOfSessions++ }
/RECI=SEND/ { currentNumberOfSessions-- }
{
        if(currentNumberOfSessions > highWaterMark){
                highWaterMark=currentNumberOfSessions
        }
}
END{
        print "Concurrent sessions high water mark = " highWaterMark
}'
}
Put the above shell function definition either in your .profile or just paste it into a terminal session. You will need to be in the work directory for your UNIX node where the statistics files are. Then you can run it as follows by piping whatever period of statistics files you want through the watermark shell function:
[work/CD.UNIX] $ cat S201712* | watermark
Concurrent sessions high water mark = 8
Now you will know whether the number of sessions you use is appropriate for your licensing of these nodes.



Monday, December 24, 2012

Self Sign


Day 19

If you're using SSL/TLS digital certificates with Connect:Direct Secure+ when connecting to your 3rd parties, you might well have a security policy that states that you will not connect with machines from 3rd parties that use self signed certificates.

You probably would prefer to connect with a 3rd party node that presents a digital certificate that is signed by someone you trust, such as VeriSign.

It might be that the history of the configuration of the Connect:Direct connection and the security policy did not coincide.

You may be in need of finding out which of your existing connections use self signed certificates.

As usual the information is within the Connect:Direct statistics, but not viewable from the "select statistics" command from  within the Connect:Direct command line.

The following function will check the statistics records piped through it to check which Secure+ connections use certificates that were signed by themselves.  In other words it checks statistics records where Secure+ was used and where the certificate issuer is the same as the subject of the server certificate used by the connection.

function selfsigncerts
{
grep RECI=CTRC | awk -F\| '
{
record["CSPE"]=""
record["CERI"]=""
record["CERT"]=""

for(i=1;i<=NF;i++)
{
key=substr($i,0,index($i,"=")-1)
value=substr($i,index($i,"=")+1)
record[key]=value
}

name = record["PNOD"] ":" record["SNOD"]

if((record["CSPE"] == "Y") && (record["CERI"] == record["CERT"]))
{
connections[name]=record["CERI"]
}
}
END{
for (name in connections)
{
print name ":" connections[name]
}
}'
}

It is used like this, assuming you are in the work directory where the statistics files are:

$ cat S20121224.001 | selfsigncerts
unx.node:OTHER.NODE:(/C=GB/L=Lincoln/O=Bank/OU=IT/CN=OTHER.NODE/emailAddress=joe.blogs@bank.co.uk/SN=12345678)

Now you know which connections use self signed certificates you can go about getting them replaced with certificates you can trust via your trusted 3rd party such as VeriSign.

Other things you could check for are the encryption algorithms used by a connection. Over time encryption algorithms lose favour as they are considered weaker than others.

Certificate signing algorithms also need checking for compliance with security policies.  For example the MD5 checksum algorithm in the past was used for signing certificates, but is considered weak, and has been shown that it can be exploited.

You may have a security policy that states you don't use certain algorithms, and you may have to demonstrate that you don't use them, and if you do, identify them for remediation.

The next few blog entries will cover these issues.

Friday, October 15, 2010

Difficult C:D questions?

Day 14


Connect:Direct records almost everything in the Connect:Direct statistics.  The statistics can be queried by using the Connect:Direct command line program “direct”, or more friendly programs such as the Connect:Direct Requester, or the browser interface, and even Sterling Control Center (SCC).

The statistics can be queried for information regarding a particular process or in general for errors and even for evidence of compliance to standards etc.

Some queries however are not possible to express in the Connect:Direct command line and difficult in the other tools mentioned earlier.

Some of these more difficult queries are listed below:

  • What is the maximum number of concurrent sessions being used, at what time and which nodes had the lion share of the sessions?
  • Which transfers are not secured using Secure+?
  • Which nodes are using self signed digital certificates?
  • What was the total volume of data transferred ordered by remote node?
  • When is the least busiest time on this Connect:Direct node?
  • Which remote nodes are triggering local scripts/processes?
  • What are the transfers that have had a failure, but have not been successfully transmitted later?

The reason why these and other queries are difficult to express in the Connect:Direct command line is that the Connect:Direct statistics contain information that you can not get at with the Connect:Direct command line.

If you have ever taken a look at the Connect:Direct statistics files on UNIX you might not have liked what you saw:



STAR=20100902 17:00:03|PNAM=PULL|PNUM=98765|SSTA=20100902 17:00:03|STRT=20100902 17:00:03|STOP=20100902 17:00:03|STPT=20100902 17:00:03|SELA=00:00:00|SUBM=aaaacd@unx.aaaa|SBID=aaaacd|SBND=unx.aaaa|SNOD=CD.OTHER|CCOD=0|RECI=CTRC|RECC=CAPR|TZDI=3600|MSGI=SCPA000I|MSST=Copy step successful.
:
etc.



The above Connect:Direct statistic record is for a COPY statement within a Connect:Direct process.  It is just one long line with all the fields separated by the ‘|’ character.  Each field contains the 4 character name for the field name, an equals sign followed by the value of that field.

The 4 letter field names are documented in the “Connect:Direct for UNIX User Guide”.  You do not need to know them all.  Just use the ones you need when you need them.



PNAM is Process Name
PNUM is Process Number
PNOD is Primary Node Name
SNOD is Secondary Node Name
CCOD is Condition Code
SFIL is Source File Name
DFIL is Destination File Name
DBYW is Destination Bytes Written



A simple UNIX command can help make these statistics files easier to read

$ cat S20100902.047 | grep RECI=CTRC | tr '|' '\n'

The grep for records that contain the string “RECI=CTRC” filters just those records that are Copy Termination ReCords i.e. produced by a COPY statement.

Which produces something like the following:



STAR=20100902 17:00:03
PNAM=PULL
PNUM=98765
:
SUBM=aaaacd@unx.aaaa
:
SNOD=CD.OTHER
CCOD=0
RECI=CTRC
:
MSGI=SCPA000I
MSST=Copy step successful.
:
PNOD=unx.aaaa
SNOD=CD.OTHER
LNOD=P
:
CSPE=Y
CSPP=TLSv1
CSPS=TLS_RSA_WITH_AES_256_CBC_SHA
CERT=(/C=GB/ST=Cheshire/L=Congleton/O=A Global Financial Institution Plc/OU=Middleware/CN=CD.OTHER/SN=69009876789765456787654567667729)
CERI=(/O=Trusted Network/OU=Trusted, Inc./OU=Trusted CA/OU=www.trusted.org/SN=78ee48de185b2071c9c9c3b51d7bddc1)
SFIL=\share123\outgoing\AAAAAA.123456.DAT
:
DFIL=/data/projectx/from_agfi/AAAAAA.123456.DAT
:
DBYW=1161
:
etc.



Now we know the format of the stats records and we have an easier way to view them we can write some shell functions to help us with some of our tasks with Connect:Direct on UNIX and even with these more difficult questions.

Next we will look at one of those questions and how it can be answered in more detail.

Sunday, April 4, 2010

Allowing Connections

Day 6

Earlier I explained how to make a connection and send a file to another Connect:Direct node, in this case a 3rd party. I didn't explain the steps that the remote node administrator makes in order to allow your connection to him.

First I will explain how this was done to allow in this case a remote connection to a UNIX Connect:Direct node. In the next post I will explain how this is done in the opposite direction i.e. from the UNIX machine to the local Windows Connect:Direct node.

Most of the configuration of Connect:Direct on a UNIX machine is done by editing files. The files we will be talking about today are the netmap.cfg & userfile.cfg files. Conceptually the information in the Windows Connect:Direct configuration is very similar to that stored in the UNIX Connect:Direct files. It is just the presentation of the information that is different. This is a common theme when dealing with Connect:Direct on different platforms.

Here is the netmap entry for the Windows Connect:Direct node NICKE in the UNIX Connect:Direct configuartion file netmap.cfg :
NICKE:\
:comm.info=192.168.10.10;1364:\
:comm.transport=tcp:\
:contact.name=:\
:contact.phone=:\
:descrip=:


The netmap is used to associate a network address with a Connect:Direct node name. So here you can see that the node name NICKE is associated with the ip-address 192.168.10.10 and the port 1364.  The lines in bold are the important ones; the other lines are descriptive.

The format of the configuration entry is important. You can see that field names are followed by an "=" sign and then the value for that field.
Each of these field/value pairs are surrounded by colons, and that each line ends with "\" except the last line of the configuration entry.

I talk about configuration entry, as the same format is used by other Connect:Direct UNIX configuration files such as the userfile.cfg which I'll talk about next.

*@NICKE:\
:local.id=appusr1:\

:pstmt.upload=y:\
:pstmt.upload_dir=/home/appusr1/data/outbound:\  
:pstmt.download=y:\
:pstmt.download_dir=
/home/appusr1/data/inbound:\
:descrip=:


Above you can see that it does indeed have a similar format to the netmap.cfg file, but the meaning is different. Here the first line says any user ("*") from ("@") the remote node name "NICKE" is associated with the following information.  While this might be OK for testing, in practice you should use a specific remote Connect:Direct user.  In a later post I'll go into more detail about that.

When a file is transferred from NICKE to CD.REMOTE the file is stored somewhere on the file system and is owned by some user on the system. In this case the owner of the files will be "appusr1", and the file will be stored under the "/home/appusr1/data" directory.

So you can see that the netmap entry is used to make the connection between Connect:Direct nodes and the userfile entry is used to say who can connect and where the files should live and who they will be owned by.

You can think of the following entry for the local user as default values for the above definition which is often referred to as a "proxy" user definition.
So if you do not override a field in the proxy definition the value will be taken from the local user definition pointed to by the "local.id" field in the proxy definition.
appusr1:\  
:admin.auth=n:
\
:pstmt.copy.ulimit=y:\
:pstmt.upload=y:\
:pstmt.upload_dir=:\
:pstmt.download=y:\
:pstmt.download_dir=:\  
:pstmt.runjob=y:\
:pstmt.runtask=y:\
:pstmt.run_dir=:\
:pstmt.submit_dir=:\
:pstmt.copy=y:\
:cmd.submit=y:\
:cmd.stopnmd=n:\
:cmd.trace=n:\
:cmd.chgproc=n:\

:name=:\
:phone=:\
:descrip=:\
:snode.ovrd=y:


I will not explain all the fields as the "Connect:Direct for UNIX Administration Guide" does a good job of that.

I will say that it is a good idea to lock down the proxy definition with specific upload/download/run/submit directories, and that the local user associated with the proxy definition should not be a privileged user such as root or the user who acts as the administrator of Connect:Direct.

So the next post will describe how to allow transfers in the opposite direction on the Windows Connect:Direct node NICKE.