Multiple Tomcat instances

Had to start two instances of Tomcat on the same VM (Linux) – Did the following without even knowing whether the steps were correct or not – will check later – but it worked.

Unzipped the tomcat zip file into two folders

In the second folder made the following changes to server.xml in conf/ folder

<Server port="8005" shutdown="SHUTDOWN">
to
<Server port="8006" shutdown="SHUTDOWN">
<Service name="Catalina">
to
<Service name="Catalina2">
<Connector port="8080" protocol="HTTP/1.1"  connectionTimeout="20000" redirectPort="8443" />
to 
<Connector port="8081" protocol="HTTP/1.1"  connectionTimeout="20000" redirectPort="8444" />
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
to
<Connector port="8010" protocol="AJP/1.3" redirectPort="8444" />
<Engine name="Catalina" defaultHost="localhost">
to
<Engine name="Catalina2" defaultHost="localhost">
Posted in Uncategorized | Comments Off on Multiple Tomcat instances

Oracle Java 8 – Debian

Wanted to install Oracle Java on a new Wheezy system – On searching the net got the following information which worked for me

apt-get install software-properties-common python-software-properties
add-apt-repository ppa:webupd8team/java

Edit /etc/apt/sources.list.d/webupd8team-java-wheezy.list and replace “wheezy” with “trusty”

apt-get install oracle-java8-installer

and that’s all!

Posted in Uncategorized | Tagged , , , , , | Comments Off on Oracle Java 8 – Debian

Neo4J – Indexes, Constraints

Cypher allows the creation of indexes over a property for all nodes that have a given label.   Neo4j CQL has provided “CREATE INDEX” command to create indexes on NODE’s or Relationship’s properties.

CREATE INDEX ON :Person(name)
CREATE INDEX ON :Customer (name)

Neo4j CQL has provided “DROP INDEX” command to drop an existing index of a NODE’s or Relationship’s property.

DROP INDEX ON :Person(name)

There is usually no need to specify which indexes to use in a query, Cypher will figure that out by itself.   If you want Cypher to use specific indexes, you can enforce it using hints.

MATCH (liskov:Scientist { name:'Liskov' })-[:KNOWS]->(wing:Scientist)-[:RESEARCHED]->(cs:Science { name:'Computer Science' })<-[:RESEARCHED]-(conway:Scientist { name: 'Conway' })
USING INDEX liskov:Scientist(name)
RETURN liskov.born AS column

Note that adding a unique property constraint on a property will also add an index on that property, so you cannot add such an index separately. Cypher will use that index for lookups just like other indexes. If you drop a unique property constraint and still want an index on the property, you will have to create the index.

To create a constraint that makes sure that your database will never contain more than one node with a specific label and one property value, use the IS UNIQUE syntax.

CREATE CONSTRAINT ON (book:Book) ASSERT book.isbn IS UNIQUE

By using DROP CONSTRAINT, you remove a constraint from the database.

DROP CONSTRAINT ON (book:Book) ASSERT book.isbn IS UNIQUE
To create a constraint that makes sure that all relationships with a certain type have a certain property, use the ASSERT exists(variable.propertyName) syntax.
REATE CONSTRAINT ON ()-[like:LIKED]-() ASSERT exists(like.day)
DROP CONSTRAINT ON ()-[like:LIKED]-() ASSERT exists(like.day)

Posted in Uncategorized | Tagged , | Comments Off on Neo4J – Indexes, Constraints

Neo4J – Functions

[table id=4 /]

Posted in Uncategorized | Tagged | Comments Off on Neo4J – Functions

Neo4J – Delete and Remove

Main Difference between Neo4j CQL DELETE and REMOVE commands –

  • DELETE operation is used to delete Nodes and associated Relationships.
  • REMOVE operation is used to remove labels and properties.

Similarity between Neo4j CQL DELETE and REMOVE commands –

  • Both commands should not be used as alone.
  • Both commands should be used with MATCH Command.

Samples

DELETE a Node clause syntax
DELETE <node-name-list>

We should use comma(,) operator to separate the node names.

Sample

MATCH (e:Employee) DELETE e
DELETE Node and Relationship clause syntax
DELETE <node1-name>,<node2-name>,<relationship-name>

Sample

MATCH (cc: CreditCard)-[rel]-(c:Customer) 
DELETE cc,c,rel
REMOVE Property clause syntax
REMOVE <property-name-list>
<property-name-list> syntax
<node-name>.<property1-name>,
<node-name>.<property2-name>, 
.... 
<node-name>.<propertyn-name>
  • We should use comma(,) operator to separate the label names list.
  • We should use dot(.) operator to separate the node names and label name.

Samples

MATCH (book { id:5 })
REMOVE book.price
RETURN book

It is similar to the below SQL Command.

ALTER TABLE BOOK REMOVE COLUMN PRICE;
SELECT * FROM BOOK WHERE ID = 5;
MATCH (dc:DebitCard) REMOVE dc.cvv RETURN dc
REMOVE a Label clause syntax:
REMOVE <label-name-list>
<label-name-list> syntax
<node-name>:<label1-name>,
<node-name>:<label2-name>, 
.... 
<node-name>:<labeln-name>
  • We should use comma(,) operator to separate the label names list.
  • We should use colon(:) operator to separate the node names and label name.

Sample

MATCH (m:Movie) REMOVE m:Picture
Posted in Uncategorized | Tagged , | Comments Off on Neo4J – Delete and Remove

Ne04J – Labels / WHERE

A label is a named graph construct that is used to group nodes into sets; all nodes labeled with the same label belongs to the same set. Many database queries can work with these sets instead of the whole graph, making queries easier to write and more efficient to execute. A node may be labeled with any number of labels, including none, making labels an optional addition to the graph.

Labels are used when defining constraints and adding indexes for properties

Single Label to a Node
CREATE (<node-name>:<label-name>)
Multiple Labels to a Node
CREATE (<node-name>:<label-name1>:<label-name2>.....:<label-namen>)
Labeling a relationship
CREATE (<from-node-name>:<from-label-name>)-        [(<relationship-name>:<relationship-label-name>)]     ->(<to-node-name>:<to-label-name>)
Simple WHERE clause syntax
WHERE <condition>
Complex WHERE clause syntax
WHERE <condition> <boolean-operator> <condition>

We can put multiple conditions on same command by using Boolean Operators (AND OR NOT XOR).

<condition> syntax:
<property-name> <comparison-operator> <value>
Operators <, >, <=, >=, <>, =

Examples

MATCH (emp:Employee) WHERE emp.name = 'Abc' RETURN emp
MATCH (emp:Employee) WHERE emp.name = 'Abc' OR emp.name = 'Xyz' RETURN emp
Create a Relationship between two existing nodes
MATCH (<from-node-label-name>:<from-node-name>),(<to-node-label-name>:<to-node-name>) WHERE <condition> CREATE (<from-node-label-name>)-[<relationship-label-name>:<relationship-name>       {<relationship-properties>}]->(<to-node-label-name>)

 

Posted in Uncategorized | Tagged , | Comments Off on Ne04J – Labels / WHERE

Neo4J – Relationships

In Neo4j, Relationships between two nodes are directional. They are either Uni-Directional or Bi-Directional. As Neo4j follows Property Graph Data Model, it should support only directional relationships.

CREATE (<node1-details>)-[<relationship-details>]->(<node2-details>)

If we observe the above syntax, it is using an arrow mark: ( ) – [ ] → ( ) . It denotes a direction from left side node to right side node.

Relationships without Properties – Existing nodes
MATCH (<from-node-label-name>:<from-node-name>),(<to-node-label-name>:<to-node-name>)
CREATE (<from-node-label-name>)-[<relationship-label-name>:<relationship-name>]->(<to-node-label-name>)
RETURN <relationship-label-name>
 Relationship with Properties – Existing Nodes
MATCH (<from-node-label-name>:<from-node-name>),(<to-node-label-name>:<to-node-name>)
CREATE(<from-node-label-name>)-[<relationship-label-name>:<relationship-name>{<define-properties-list>}]->(<to-node-label-name>)
RETURN <relationship-label-name>
Relationship without Properties – New Nodes
CREATE(<from-node-label-name>:<from-node-name>)-   [<relationship-label-name>:<relationship-name>]->   (<to-node-label-name>:<to-node-name>)
RETURN <relationship-label-name>
Relationship With Properties With New Nodes

CREATE (<from-node-label-name>:<from-node-name>{<define-properties-list>})-[<relationship-label-name>:<relationship-name>{<define-properties-list>}]->(<to-node-label-name>:<to-node-name

Neo4j – Retrieve Relationship Node’s Details
MATCH (<from-node-label-name>)-[<relationship-label-name>:<relationship-name>]->(<to-node-label-name>)
RETURN <relationship-label-name>

 

 

 

 

 

 

Posted in Uncategorized | Tagged | Comments Off on Neo4J – Relationships

Neo4J – A Quick Look

Features / Advantages
  • SQL Like easy query language Neo4j CQL
  • It follows Property Graph Data Model
    • A graph records data in nodes and relationships. Both can have properties. This is sometimes referred to as the “Property Graph Model”.
  • It supports Indexes by using Apache Lucence
  • It supports UNIQUE constraints
  • It contains a UI to execute CQL Commands : Neo4j Data Browser
  • It supports full ACID(Atomicity, Consistency, Isolation and Durability) rules
  • It uses Native graph storage with Native GPE(Graph Processing Engine)
  • It supports exporting of query data to JSON and XLS format
  • It provides REST API to be accessed by any Programming Language like Java, Spring,Scala etc.
  • It provides Java Script to be accessed by any UI MVC Framework like Node JS.
  • It supports two kinds of Java API: Cypher API and Native Java API to develop Java applications.

[table id=2 /]

A traversal navigates through a graph to find paths.  Traversing a graph means visiting its nodes, following relationships according to some rules. In most cases only a subgraph is visited

 

[table id=3 /]

Examples
 CREATE (<node-name>:<label-name>)

 CREATE (<node-name>:<label-name>
   { 
     <Property1-name>:<Property1-Value>,
     ........,
     <Propertyn-name>:<Propertyn-Value>
   }
 )
MATCH (<node-name>:<label-name>)
RETURN 
 <node-name>.<property1-name>,
 ........
 <node-name>.<propertyn-name>
Posted in Uncategorized | Tagged | Comments Off on Neo4J – A Quick Look

Informative Links

[table id=1 /]

Posted in Uncategorized | Tagged | Comments Off on Informative Links

Threads – Server Freeze

The issue
While working on one of the projects, the client had reported an issue that the memory allocation to the Java process keeps increasing over the period of time and roughly after 2-3 weeks users cannot login to the application.
About the application
It was an Element Management System, where periodically the status of each device managed by the EMS would be checked for by starting one thread per device.  Apart from the same at period intervals the traffic information associated to all the ports in the device was fetched from the device periodically.  There were also scheduled jobs which would backup the current configuration extracted from the device to remote system over FTP.  Added to all this users can perform configuration change operations either on one of a group of nodes in bulk (for which one thread would be started per device). The application was deployed as SARs on JBoss while clients connect over JBoss-Remoting.   Traps were received by the application from various devices which were categorized, handled and in some cases forwarded.
Challenges
Hardly any logging statements were in the code.  Reported performance issues could not be replicated always apart for slowness after 2-3 weeks and server not responding after around a month.
Observations
First things first, used jConsole to monitor the heap usage – A perfect saw tooth pattern indicating no memory leaks and max usage of ~400MB right from day 1 to day 24 when slowness started degrading the performance rapidly.  At this point access the only web page index.html – served immediately and also the download link to download client always served the zip file pretty fast.  However OS memory allocation always increasing from day 1 gradually and rapidly from around day 17.
The threads graph in jConsole gave an idea on probable root cause.  The number threads was increasing gradually from day 1 and rapidly from around day 17.
Initial conclusion
Performance degradation was not due to memory leaks.  Also the Xmx value was 4G and the memory usage reported by OS was around 9G, more over heap usage graphs peaked only to 400MB.
It was clear from the active threads graphs that the count was increasing and not dipping down.  Looking at the state of threads it was found many was were in WAIT state.  So it was more of a thread related issue and the stack space occupied could be the reason of OS level memory allocation.
The solution
The active threads were related to almost all the operations  (status polling, performance data collection, handling received traps etc).  One of the observations was that maximum number of threads were related to status polling.  As a first step added quite a few logging statements in the application code to track the start and completion of threads.
The key issue observed was that it was taking more time to complete one set/get configuration operation on a device. So when it took more than 80 seconds to respond to a specific query, 4 status polling (get system.status XML-RPC command) threads would be queued up to be executed and 1 get port.counter command would be queued up.  Certain devices took more time to respond to port.counters leading to more and more threads getting queued up.  As a first step, intelligence added to the status polling module in such a way that if any command is already being executed, treat that the status is ‘up’ and do not start a thread for the polling.
This helped a lot and the application performance degradation still happened but after 3-4 weeks and was usable even after 6 weeks.  At this point looking at the active threads it was clear that handling of traps receied threads were getting queued up.  Looking into the code, it was identified that a SingleThreadedExecutor was used for processing traps.  This was replaced with one SingleThreadedExecutor per device.  With this update the slowness issue had disappeared the max memory allocation noted from taskmgr or top commands remained at a max of 1G consistently for longer periods.
Posted in Uncategorized | Tagged , , , , , , | Comments Off on Threads – Server Freeze