HistDiff Processesor: A Local HistDiff pipeline

by Derfel Terciano

Project Status

Download

NOTE: BEFORE INSTALLING APPLICATION MAKE SURE TO TURN OFF YOUR ANTIVIRUS (WINDOWS) WHEN USING THIS APP!! I DO NOT HAVE A CERTIFICATE FOR THE APP SO IT WILL BE RAISED AS A POTENTIAL VIRUS!

What this is

This is a localized version of the full HistDiff pipeline for Cytological Profiling. The idea here is to make it easier for labs to process their own data locally. This means no web servers, no middle man for data, just pure HistDiff processing at your computer any where!

Planned features:

Due: to the ambitiousness of the project, it is difficult to implement everything

• Heat map visualization! [Completed!]

  • See your HistDiff ASAP!

• Score distribution [Not yet implmented / WIP]

  • Visualize the distribution of the HistDiff scores.

• Quality control [Deprecated]

  • Visualize the activity levels of your controls.
  • Identify any striping or plate effects in assay plate.
  • Re-run histdiff if any striping is seen on plates.
  • See PCA loadings of controls and experiment.

• Once satisifed with results, users can export their experiements.

• Planned fun feature:

  • Multiplate processing

TO-DO:

• Implement user input/config
  • File selection
  • Control definitions
    • create a pop out to select negative controls
    • construct plate assay replica
    • implement option for 96-well plate (optional)
    • Add functionality for persistent well selection state
    • TODO: fix styling
  • Additional meta information to ignore
  • Well name column
    • Implement drop down option
      • Use the following: carbon components
• Implement Rust to Svelte API
  • Rewrite HistDiff input to "mimick" histdiff binary
    • RUST SIDE IMPLEMENTATION
      • Implement formatter to make data acceptable
      • Alogrithm was re-written to histdiff_core. The input is raw cell data, processing is done on the fly
      • [ ] Output formatted data to a temp file
        • (Alternatively, try and hold data into memory and pass it into HD?) This doesn't seem like a good idea but this the only way i can think of to not produce a temp file
      • [ ] Modify rust HD to accept temp file OR modify HD to take in raw DataStruct
    • Clear app states when submitting
• Implement logging
  • Implement logging in rust processes
  • Add global emit and listening for logging
  • Prevent main UI interaction when process is running
  • Implement persistent logging
  • Modify rust hd core to use infos instead of traces
• Implement User Output functionality
  • Store the histdiff output globally in the rust backend or use a different approach
  • Implement UI for user to output HistDiff data .
    • Call a file dialogue for users to choose where to output data
• Implement visualization and visual analysis
  • Implement Heatmap view
    • Use: HTMLCanvas for heatmaps. D3 doesn't work well with large data
    • Implment heatmap
    • Functionality
      • Add zoom capabilities
      • Fit entire heatmap onto entire canvas
      • Fix dendrogram trees so it has highest quality
      • Add zoom in and out button
      • fix canvas so it sits nicely in window
      • add tool tips hover so uses can see cell info
      • Mark negative control rows with a red background\
      • Add a slide to adjust contrast
    • Implement dendrograms
      • Left dendrogram
      • Right dendrogram
    • Use new clustering library
    • Integrate clustering library to app's rust library
      • Convert json tree into D3 compliant json tree
      • Implement dendrogram
      • Create ability to safely transpose dataframes so both rows and cols are clustered
        • This needs to be an option if users want to see features clustered
      • Parse backend output for svelte to use
    • Implement backend aspect of the heatmap
      • API has been implemented, dev can now call get_cluster_res to grab d3 results
    • Implement Score distribution
  • Implement positive control score distribution

Major Bugs:

• HistDiff panicks at only 1 cntrl well selected.
  • ISSUE: modify UI to format the control and well names to remove any leading zeroes.
  • Format well control input so histdiff doesn't panic.
• Address Error handling
  • ISSUE: emit a separate error flag to soft reset app state
  • Implement error emits
• Fix Feature label mis-alignment