Notes from the Trenches

Last Updated: 9/7/2019

 

AWS Marketplace makes it easy to launch Graphistry. This tutorial shares a visual guide of launching Graphistry, trying your first notebook, and help with common configuration options and occasional error handling.  When you're ready, head to AWS Marketplace! And, if you haven't heard about Graphistry's  AWS launch, learn about why we did it and what's new.

 

Launch a private Graphistry instance from AWS Marketplace

 

 

  

1. Subscribe Screen

• If prompted, accept terms and wait through "Pending" period
  • Do not configure the annual contract discounted prepay option: Total contract price should be $0
• Continue to Configuration

 

2. Configure Screen

• Version: Default - the latest Graphistry version
• Region: Pick a GPU-capable region close to your users or data
  • Tip: Search for P3 series (Nvidia V100)  on the EC2 instance comparison chart, with US East and Oregon as great options in the US.
  • Do not configure the annual contract discounted prepay option: Total contract price should be $0
• Continue to Launch

 

3. Launch Screen

• Action: Default - Launch from Website
• Instance: Default is recommend - p3.2xlarge is the smallest Nvidia-RAPIDS-compatible GPU
• Security group: We recommend "Create new based on Seller Settings"
• Key pair: Use existing if you have a copy of the files, else create new (and save), and likely  `chmod 400 myfile.pem` (or chmod 644 myfile.pem)
• Launch

• Wait till "Status Checks: 2/2 checks"
• Follow the public IP / DNS link from your EC2 console to start your first Graphistry session!

 

Login & launch your first visualization & notebook

 

 

 

1. Login

• In a browser, go to the public DNS provided by the EC2 console
• Upon starting the system, you need to wait ~5 minutes, and until then, may see 5XX errors or otherwise incomplete page loads
• As a first-time user, create the initial admin account from the webpage, and be sure to record the email/user/pwd

 

2. Open a notebook

• "Analyst Getting Started" -> "See Notebook"
• In sequence, run each code cell by clicking on it and entering "shift-enter"
• On cell "g.plot()", on the upper right of the graph where it says "workbook", click the left-most icon for "Open in a new window", and click the Graphistry logo to begin
• Follow the UI guide on how to interact with the notebook

 

3. Try more notebooks

• Try other notebooks: for developers, CSV upload of your own data, various DBs, ...
• Make a copy of a notebook you want to use as a template 
• You can always edit any notebook cell, re-run it, and save your changes/outputs
• Best practice is to make sure you can "run-all" in a notebook and it gives the full results

 

Recommended additional config & troubleshooting:

See the Graphistry AWS Marketplace FAQ:

• Errors: Handle common AWS issues like no available GPUs
• IP/Domain: Set an elastic IP or domain to have a stable URL across restarts
• Enable Graphistry notebooks to communicate with secure databases
  • If `pip install` does not have your package, SSH into the instance, and then the Docker container as root
  • Make sure the Graphistry IP is permitted by the security group for your DB
• Create accounts for your users
• Setup TLS and restrict server access to your VPN

Next steps & further reading

• UI guide 
• Graphistry AWS Marketplace FAQ 
• Admin docs
• Python notebooks

 

With our big 2.0 release, Graphistry's tech and features have combined into a fast and easy way to connect to your team's data,  visually understand most large or complex problems, and do it all safely & privately. For the first time, you can now easily experience Graphistry from your own AWS account. Read on for what problems our early users are solving, how Graphistry 2.0 is helping them, and quickly getting started in AWS on your own data.

When everything runs on GPUs, we can fundamentally shift the way we experience data analysis much like video moving to HD or shifting from black-and-white to color. What if you could load your full dataset, ask whole-table questions like what are the patterns, and get the answers... immediately? What if you could do that visually, replacing writing queries with simple infinite zoom and direct manipulations down to the level of individual data points? Core analytics areas like security, fraud, operations, and customer 360 are entering this sci-fi-level world of rapid hypothesis iteration.

Incident responders and threat hunters are often facing a bit of an analytical catch-22. They typically have access to more and higher fidelity data sources than ever before, yet the volume and complexity of the data can often make it hard to see the point that matters.

Graph visualization has proven to be powerful for investigating almost any type of data, and most recently the team at Graphistry was able to help in uncovering a massive Ethereum heist on two of the world’s most popular DApps (distributed applications).

Learning to Whitebox the SOC-in-a-Box

Even as organizations automate their security operations with orchestration and AI, some of the most important parts of security investigations continue to depend on human analysis and talent. These critical moments in the investigation remain frustratingly slow, and need categorically different technologies that are optimized for human-in-the-loop analysis.

Security is in the midst of a transformation that is putting extreme pressure on security analysts and hunt teams. One shift that is causing teams a lot of pain in their daily work is that as threats have gotten more sophisticated, security products have gotten much less sure of themselves. Security products increasingly detect the “anomalous” and report threats on a sliding scale of confidence. Not only must staff deal with advanced threats, but they must spend an increasing amount of time navigating the grey areas and ambiguities of modern threat detections to determine and deliver the right actions.

Welcome to the  Age of Maybe, where it is critical that we arm analysts for dealing with the indicators that are diverse, widespread...and uncertain.

Today, Verizon released the 2018 edition of the Data Breach Investigation Report, and the Graphistry team is proud to have been able to contribute to the report. In fact, the cover of the report is a stylized and simplified version of our graph analysis that you can see on page 56 of the report!

The Graphistry team is excited to report: production-grade open GPU compute is coming to JavaScript with the Apache Arrow[JS] project and GOAI. We have been contributing to these projects because they are big enablers for the web. In our case, that means we can build best-of-class visual fastpaths for security and fraud teams struggling to investigate through tools like Splunk, Elastic, and Hadoop.

O'Reilly's Data Show recently had our CEO, Leo Meyerovich, on to talk about why and how enterprises and data teams are adopting graph technology. You can check it out here where we dive into how we are using graphs as an interface to AI tools & data. 

Meanwhile, our team is on the move. Let us know if you'll be near one of our upcoming talks and events - we love catching up with current & new users!