Getting Started with Hadoop in DataStax Enterprise¶

The Portfolio Manager Use Case¶

The portfolio manager application demonstrates a hybrid workflow using DataStax Enterprise. The use case is a financial application where users can actively create and manage a portfolio of stocks.

On the Cassandra OLTP (online transaction processing) side, each portfolio contains a list of stocks, the number of shares purchased, and the price at which the shares were purchased. A live stream of data simulates an active stock market, and updates each portfolio based on its overall value and the percentage of gain or loss compared to the purchase price. Historical market data is tracked for each stock (the end-of-day price) going back in time.

In the demo, simulated real-time stock data is generated by the pricer utility. This utility generates portfolios, live stock prices, and historical market data.

On the DSE OLAP side, a Hive MapReduce job calculates the greatest historical 10 day loss period for each portfolio, which is an indicator of the risk associated with a portfolio. This information is then fed back into the real-time application to allow customers to better gage their potential losses.

Running the Portfolio Manager Demo¶

Before you begin, make sure you have installed, configured, and started DSE on either a single node (as an Analytics node) or a cluster. If running the demo on a cluster, install and run the demo from the DSE Job Tracker (analytics seed) node.

1. Go to the portfolio manager demo directory.

   • Tarball install: cd <install_location>/demos/portfolio_manager
   • Packaged install: cd /usr/share/dse-demos/portfolio_manager

   Note

   You must run the pricer utility from a directory where you have write permissions (such as your home directory), or else run it as root or using sudo.

2. Run the bin/pricer utility to generate 100 days worth of historical stock data:

   • To see all of the available options for this utility:

     bin/pricer --help
     

   • If running on a single node cluster on localhost (as described in Quick Start with DataStax Enterprise):

     bin/pricer -o INSERT_PRICES
     bin/pricer -o UPDATE_PORTFOLIOS
     bin/pricer -o INSERT_HISTORICAL_PRICES -n 100
     

   • If running the demo on a cluster:

     LOCAL_IP=<demo_node_ip_address>
     bin/pricer -o INSERT_PRICES -d $LOCAL_IP \
     --replication-strategy="org.apache.cassandra.locator.NetworkTopologyStrategy" \
     --strategy-properties="Analytics:1,Cassandra:1"
     
     bin/pricer -d $LOCAL_IP -o UPDATE_PORTFOLIOS
     bin/pricer -d $LOCAL_IP -o INSERT_HISTORICAL_PRICES -n 100
     

     NetworkTopologyStrategy is the preferred replication placement strategy. For more information, see NetworkTopologyStrategy.

3. Start the web service.

   cd website
   java -jar start.jar &
   

4. Open a browser and go to http://localhost:8983/portfolio to see the real-time Portfolio Manager demo application.

   
   
   

5. Open another shell window.

6. Start Hive and run the MapReduce job for the demo in Hive.

   dse hive -f /usr/share/dse-demos/portfolio_manager/10_day_loss.q
   

   or for binary installations:

   <install_location>/bin/dse hive -f <install_location>/demos/portfolio_manager/10_day_loss.q
   

7. The MapReduce job will take several minutes to run. Open the URL http://localhost:50030/jobtracker.jsp in a browser to watch the progress in the Job Tracker.

8. After the job completes, refresh the Portfolio Manager web page to see the results of the Largest Historical 10 day Loss for each portfolio.