cloud-blockchain-book : 积累广度，增加深度
author: heidsoft
created: 2012-08-30 03:31:02
bigdata book cloud machine-learning spark tensorflow c


alluxio : Alluxio, formerly Tachyon, Unify Data at Memory Speed
author: Alluxio
created: 2012-12-21 17:43:46
alluxio big-data distributed-storage hadoop memory-speed presto spark tensorflow virtual-file-system java


RoaringBitmap : A better compressed bitset in Java
author: RoaringBitmap
created: 2013-06-17 20:29:23
bitset druid java lucene roaring-bitmaps roaringbitmap spark


scala-samples : There are pieces of scala code that explain Scala syntax and related things - like what you can do with all this
author: Sergey80
created: 2013-08-24 12:41:59
algorithms samples scala scalajs spark


hadoop_cookbook : Cookbook to install Hadoop 2.0+ using Chef
author: cdapio
created: 2013-10-23 22:56:32
chef chef-cookbook cookbooks hadoop hadoop-cookbook hbase hive spark zookeeper ruby


Spark : Cross-platform real-time collaboration client optimized for business and organizations.
author: igniterealtime
created: 2013-11-23 19:34:06
collaboration cross-platform jabber java openfire spark xmpp xmpp-client


deeplearning4j : Deeplearning4j, ND4J, DataVec and more - deep learning & linear algebra for Java/Scala with GPUs + Spark
author: deeplearning4j
created: 2013-11-27 02:03:28
artificial-intelligence clojure deeplearning deeplearning4j dl4j gpu hadoop intellij java linear-algebra matrix-library neural-nets python scala spark


nodejs-spark : Read and control your Spark Core with node.js :)
author: fvdm
created: 2014-01-11 01:17:50
api-client nodejs particle-io remote-control spark unlicense javascript


geotrellis-chatta-demo : Demo of GeoTrellis - weighted overlay and zonal summary for University of Tennessee at Chattanooga.
author: geotrellis
created: 2014-01-17 23:30:00
chattanooga geodocker-cluster geotrellis s3 spark javascript


h2o-3 : Open Source Fast Scalable Machine Learning Platform For Smarter Applications (Deep Learning, Gradient Boosting, Random Forest, Generalized Linear Modeling (Logistic Regression, Elastic Net), K-Means, PCA, Stacked Ensembles, Automatic Machine Learning (AutoML), ...)
author: h2oai
created: 2014-03-03 16:08:07
automatic big-data data-science deep-learning distributed ensemble-learning gbm gpu h2o hadoop java machine-learning multi-threading naive-bayes opensource pca python r random-forest spark


emma : A quotation-based Scala DSL for scalable data analysis.
author: emmalanguage
created: 2014-03-03 17:30:30
dsl emma flink quotations scala scala-dsl scalable-data-analysis spark


OpenLync : Instant messenger write in Java for local and corporate environment
author: RobertoDebarba
created: 2014-03-09 21:59:52
chat instant-messaging java messaging messenger microsoft-lync socket spark


wirbelsturm : Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data related infrastructure.
author: miguno
created: 2014-03-17 17:02:51
apache-kafka apache-spark apache-storm kafka puppet spark storm vagrant shell


spyrk : A python port of @holman's spark
author: mre
created: 2014-03-28 13:35:42
holman python spark


spark-scala-tutorial : A free tutorial for Apache Spark.
author: deanwampler
created: 2014-05-01 20:33:48
jupyter scala spark tutorial jupyter notebook


supaku-sukara : Functional Programming, Functional Programming Exercise Solutions in Scala & Spark
author: afsalthaj
created: 2014-05-03 14:29:37
functional-programming functor language monad parallelism scala shapeless spark typeclasses


kafka-storm-starter : Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
author: miguno
created: 2014-05-23 14:33:10
apache-avro apache-kafka apache-spark apache-storm avro integration kafka scala spark storm


ethz-web-scale-data-mining-project : ETH Zurich - Web Scale Data Processing and Mining Project
author: lukaselmer
created: 2014-06-10 18:09:27
big-data data-mining eth hadoop machine-learning scala spark topic-modeling web-scale html


Play-Spark-Scala :
author: knoldus
created: 2014-06-12 13:27:41
akka css knoldus mllib play-framework sbt scala spark


dockers : Docker hello world templates
author: doubaokun
created: 2014-06-23 10:08:14
docker docker-template kafka shell spark vagrant zookeeper


spark-cassandra-connector : DataStax Spark Cassandra Connector
author: datastax
created: 2014-06-27 15:45:54
cassandra scala spark


hbase-rdd : Spark RDD to read and write from HBase
author: unicredit
created: 2014-07-04 12:03:38
hbase scala spark


JS-Spark : Realtime calculation distributed system. AKA distributed lodash
author: syzer
created: 2014-07-24 17:00:50
distributed distributed-computing multicore realtime spark javascript


dllib : dllib is a distributed deep learning library running on Apache Spark
author: Lewuathe
created: 2014-08-02 05:29:16
deep-learning mllib scala spark css


wowauctions-server : wow auctions web/spark services http://wow.ooktioneer.com
author: sbenner
created: 2014-08-07 22:28:32
java jquery mongo-spark spark spring-boot wow


spark-jobserver : REST job server for Apache Spark
author: spark-jobserver
created: 2014-08-21 23:07:47
rest-api scala spark spark-jobserver


nd4j : Fast, Scientific and Numerical Computing for the JVM (NDArrays)
author: deeplearning4j
created: 2014-08-31 03:47:00
artificial-intelligence backend dl4j gpu java jvm nd4j numerical-calculations scala-notebook scientific scientific-computing spark


docker_practice : Learn and understand Docker technologies, with real DevOps practice!
author: yeasy
created: 2014-09-05 04:06:39
book cloud-computing container devops docker kubernetes linux mesos spark swarm dockerfile


spark-notebook : Interactive and Reactive Data Science using Scala and Spark.
author: spark-notebook
created: 2014-09-05 19:35:25
apache-spark data-science notebook reactive scala spark javascript


SparkJobServerClient : Java Client of the Spark Job Server implementing the arranged Rest APIs
author: bluebreezecf
created: 2014-09-08 15:28:05
jar java-client pom spark java


LearningSpark : Scala examples for learning to use Spark
author: spirom
created: 2014-09-28 02:18:01
scala spark spark-streaming sparkcore sparksql


grimlock : library for performing data-science and machine learning related data preparation, aggregation, manipulation and querying tasks at scale using Scalding or Spark
author: CommBank
created: 2014-10-01 03:30:53
data-preparation data-science scala scalding spark


josimtext : A system for word sense induction and disambiguation based on JoBimText approach
author: uhh-lt
created: 2014-10-21 10:41:35
count-based distributional-semantics jobimtext noun-sense-induction sense-clusters sense-disambiguation sense-induction spark scala


succinct : Enabling queries on compressed data.
author: amplab
created: 2014-11-08 23:22:20
big-data compression java scala spark succinct


gatk : Official code repository for GATK versions 4 and up
author: broadinstitute
created: 2014-12-02 20:50:36
bioinformatics dna gatk genome genomics ngs science sequencing spark java


python_moztelemetry : Spark bindings for Mozilla Telemetry
author: mozilla
created: 2014-12-09 17:24:55
data-analysis mozilla-telemetry python spark


wordcount__spark_scala :
author: jonathanhefner
created: 2015-01-08 18:29:30
demo scala spark


spark-hbase-connector : Connect Spark to HBase for reading and writing data with ease
author: nerdammer
created: 2015-01-09 15:23:09
hbase spark scala


fluo-muchos : Muchos (an Apache Fluo project) sets up Apache Accumulo or Apache Fluo on a cluster (optionally launched in Amazon EC2) for development
author: apache
created: 2015-01-21 13:23:09
accumulo amazon-ec2 fluo hadoop spark python


data-science-ipython-notebooks : Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
author: donnemartin
created: 2015-01-23 19:38:29
aws big-data caffe data-science deep-learning hadoop kaggle keras machine-learning mapreduce matplotlib numpy pandas python scikit-learn scipy spark tensorflow theano


ruby-spark : Ruby wrapper for Apache Spark
author: ondra-m
created: 2015-01-26 20:07:38
distributed rdd ruby ruby-spark spark


sparta : Real Time Analytics and Data Pipelines based on Spark Streaming
author: Stratio
created: 2015-02-04 08:04:40
analytics hdfs kafka lambda olap real-time scala spark spark-streaming sparksql sparta stratio stratio-sparta streaming streaming-data triggers workflow


seldon-server : Machine Learning Platform and Recommendation Engine built on Kubernetes
author: SeldonIO
created: 2015-02-11 13:49:35
aws azure cloud deep-learning deployment docker gcp java kafka kafka-streams kubernetes machine-learning microservices prediction python recommendation-engine recommender-system seldon spark tensorflow


silex : something to help you spark
author: radanalyticsio
created: 2015-02-12 22:24:36
scala silex spark


SparkLED : Server (Spark Core) and PC client for controlling a LED display (or strip) over WiFi. Server programmed in C (Arduino), client in Python 3
author: olesk75
created: 2015-02-21 20:43:03
arduino c emulator iot leds particles photon python spark wifi


example-spark : Spark, Spark Streaming and Spark SQL unit testing strategies
author: mkuthan
created: 2015-02-24 21:52:59
spark spark-streaming testing scala


twitter-sentiment-analysis : Streaming tweets with spark, language detection & sentiment analysis, dashboard with Kibana
author: vspiewak
created: 2015-02-28 13:46:35
dashboard kibana nlp scala sentiment-analysis spark tiwtter


spark-terasort : Spark Terasort
author: ehiggs
created: 2015-03-09 22:59:42
spark terasort java


emr-bootstrap-spark : AWS bootstrap scripts for Mozilla's flavoured Spark setup.
author: mozilla
created: 2015-03-14 08:01:33
aws jupyter spark zeppelin python


example-spark-kafka : Apache Spark and Apache Kafka integration example
author: mkuthan
created: 2015-03-17 19:22:32
kafka spark spark-streaming scala


Spark : Indie RPG with roguelike dungeons
author: bretsky
created: 2015-03-18 10:27:15
dungeon enemies game inventory python roguelike roguelike-dungeons spark terminal-emulators vision


benchm-ml : A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).
author: szilard
created: 2015-03-28 00:34:18
data-science deep-learning gradient-boosting-machine h2o machine-learning python r random-forest spark xgboost


frameless : Expressive types for Spark.
author: typelevel
created: 2015-03-28 06:03:10
fp functional-programming scala spark typelevel


spark-infotheoretic-feature-selection : This package contains a generic implementation of greedy Information Theoretic Feature Selection (FS) methods. The implementation is based on the common theoretic framework presented by Gavin Brown. Implementations of mRMR, InfoGain, JMI and other commonly used FS filters are provided.
author: sramirez
created: 2015-04-23 16:03:34
brown feature-selection filter mrmr spark scala


spark-MDLP-discretization : Spark implementation of Fayyad's discretizer based on Minimum Description Length Principle (MDLP)
author: sramirez
created: 2015-04-23 16:47:00
discretization fayyad-discretizer spark scala


spark-py-notebooks : Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
author: jadianes
created: 2015-05-06 07:45:21
big-data bigdata data-analysis data-science ipython ipython-notebook machine-learning mllib notebook pyspark python spark jupyter notebook


PyRollout : A PySpark script that builds a static site from an RDF source
author: mattkohl
created: 2015-05-06 18:35:36
python rdf spark static-site


fast-mRMR : An improved implementation of the classical feature selection method: minimum Redundancy and Maximum Relevance (mRMR).
author: sramirez
created: 2015-05-07 11:38:20
fast feature-selection mrmr spark c++


Spark.jl : Julia binding for Apache Spark
author: dfdx
created: 2015-05-25 22:33:25
big-data julia spark


magellan : Geo Spatial Data Analytics on Spark
author: harsha2010
created: 2015-06-01 01:06:52
big-data geojson geometric-algorithms geospatial geospatial-analysis geospatial-analytics geospatial-processing magellan shapefile spark sparksql scala


spark-knn-graphs : Spark algorithms for building k-nn graphs
author: tdebatty
created: 2015-06-08 06:43:21
algorithm knn-graphs lsh-superbit nearest-neighbor-search nn-descent processing-knn-graphs spark spark-knn-graphs html


piglet : A compiler for Pig Latin to Spark and Flink.
author: dbis-ilm
created: 2015-06-10 06:26:28
flink pig pig-compiler pig-latin scala spark


sbt-spark-submit : sbt plugin for spark-submit
author: saurfang
created: 2015-06-17 04:13:59
sbt spark scala


camus-compressor : Camus Compressor merges files created by Camus and saves them in a compressed format.
author: allegro
created: 2015-06-18 12:15:30
avro etl hadoop kafka spark java


Apache-Spark-Starter-Kit : Easy tool for Apache Spark Starter
author: peterlee0127
created: 2015-06-22 02:07:05
apache-spark docker python scala spark


asakusafw-spark : Asakusa on Spark
author: asakusafw
created: 2015-06-22 08:18:00
asakusa-framework batch batch-processing big-data spark scala


spark-twitter-stream-example : "Sentiment analysis" on a live Twitter feed with Apache Spark and Apache Bahir
author: stefanobaghino
created: 2015-06-25 16:19:29
spark streaming twitter scala


spark-streaming-gnip : An Apache Spark utility for pulling Tweets from Gnip's PowerTrack in realtime
author: knoldus
created: 2015-06-29 04:09:20
gnip gnip-powertrack knoldus pulling-tweets realtime scala spark spark-streaming spark-utility sparkconf sparkcontext tweets


spark-movie-lens : An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset
author: jadianes
created: 2015-07-02 07:04:17
big-data bigdata flask movie-recommendation movielens-dataset python spark jupyter notebook


spark-tsne : Distributed t-SNE via Apache Spark
author: saurfang
created: 2015-07-06 19:01:28
spark tsne scala


pipeline : PipelineAI: Real-Time Enterprise AI Platform
author: PipelineAI
created: 2015-07-08 03:49:23
airflow artificial-intelligence cassandra docker elasticsearch gpu kafka kubernetes machine-learning microservices netflixoss neural-network nifi pipeline prediction presto redis scikit spark tensorflow java


dev-setup : macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
author: donnemartin
created: 2015-07-08 21:59:38
android-development aws bash cli cloud elasticsearch git iterm2 linux mac macos mongodb mysql nodejs postgresql python redis spark sublime-text vim


spark-druid-olap : Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.
author: SparklineData
created: 2015-07-09 02:00:39
business-intelligence olap-cube query-optimization spark sparksql scala


aws-kinesis-scala : Scala client for Amazon Kinesis. Also provides write to Kinesis capability for Apache Spark or Spark Streaming.
author: bizreach
created: 2015-07-13 05:31:06
apache-spark aws aws-kinesis kinesis scala spark


scala-basics : Test_python amm
author: sivaprakashniet
created: 2015-07-14 07:06:04
jobse playframework spark scala


latest-jupyter-skel :
author: CraigKelly
created: 2015-07-15 23:58:25
docker-image jupyter notebook python spark jupyter notebook


quill : Compile-time Language Integrated Queries for Scala
author: getquill
created: 2015-07-20 22:15:04
cassandra database jdbc linq mysql postgres scala scalajs spark sparksql


azure-event-hubs : ☁️ Cloud-scale telemetry ingestion from any stream of data with Azure Event Hubs
author: Azure
created: 2015-07-21 15:32:18
amqp apache azure c dotnet event-hubs eventhub eventhubs go golang java messaging microsoft node node-js nodejs python spark stream streaming


data-processing-pipeline : Real-Time Data Processing Pipeline & Visualization with Docker, Spark, Kafka and Cassandra
author: rogaha
created: 2015-07-22 10:25:08
cassandra digital-ocean docker-machine kafka spark twitter twitter-streaming-api visualization html


pucket : Bucketing and partitioning system for Parquet
author: intenthq
created: 2015-07-27 09:42:20
hdfs parquet partitioning scala spark thrift


yuzhouwan : Code Library for My Blog
author: asdf2014
created: 2015-08-04 06:32:09
algorithm bigdata druid elasticsearch hadoop hbase nio spark tensorflow zookeeper java


ArchiveSpark : An Apache Spark framework for easy data processing, extraction as well as derivation for Web archives and archival collections, developed by the Internet Archive and L3S Research Center.
author: helgeho
created: 2015-08-06 19:42:53
archivespark internet-archive spark spark-framework warc web-archiving webarchive jupyter notebook


aliyun-emapreduce-sdk : Hadoop/Spark on Aliyun, supporting interactions with Aliyun's base services.
author: aliyun
created: 2015-08-12 08:34:04
aliyun e-mapreduce kafka spark java


elephas : Distributed Deep learning with Keras & Spark
author: maxpumperla
created: 2015-08-13 12:09:19
deep-learning distributed-computing keras neural-networks spark python


AssortmentOfJUnitRules : An assortment of (for me anyway) useful JUnit rules.
author: mlk
created: 2015-08-17 13:39:45
component-testing dynamodb dynamodb-local hacktoberfest hadoop integration-testing junit-rule mocking mongo s3 sftp-server spark stubbing java


azure-event-hubs-spark : Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs
author: Azure
created: 2015-09-04 05:11:45
apache apache-spark azure connector continuous databricks event-hubs eventhubs microsoft real-time scala spark spark-streaming stream streaming structured-streaming


spark-knn : k-Nearest Neighbors algorithm on Spark
author: saurfang
created: 2015-09-09 04:49:45
knn spark scala


spark-google-spreadsheets : Google Spreadsheets datasource for SparkSQL and DataFrames
author: potix2
created: 2015-09-10 11:20:38
data-frame scala spark sparksql spreadsheet


iis : Information Inference Service of the OpenAIRE system
author: openaire
created: 2015-09-11 05:55:08
big-data data-mining data-processing-system hadoop iis information-inference openaire spark text-mining java


Cook : Fair job scheduler on Mesos for batch workloads and Spark
author: twosigma
created: 2015-09-11 18:20:08
cluster mesos scheduler spark clojure


Ginger : Ginger - Opinionated RESTful Routing powered by Spark
author: bradcypert
created: 2015-09-16 04:41:18
framework ginger java spark


snappydata : SnappyData - The Spark Database. Stream, Transact, Analyze, Predict in one cluster
author: SnappyDataInc
created: 2015-09-16 10:36:36
analytics memory-database scale snappydata spark stream transaction scala


Spark_Streaming_Pipeline : Proof of Concept for Apache Spark ,Apache Kakfa and MongoDB Integration
author: saurabhsharma721
created: 2015-09-18 02:23:35
kafka mongodb mongodb-java-connectors spark java


sparkmagic : Jupyter magics and kernels for working with remote Spark clusters
author: jupyter-incubator
created: 2015-09-21 15:35:25
cluster jupyter jupyter-notebook kerberos kernel livy magic notebook pandas-dataframe pyspark spark sql-query python


utils4s : scala、spark使用过程中，各种测试用例以及相关资料整理
author: jacksu
created: 2015-09-24 09:15:03
akka breeze json4s scala scala-demo scala-spark spark spark-streaming


BigLibrary :
author: recipegrace
created: 2015-09-25 19:59:18
electric spark scala


ml-vm-notebook : Machine Learning Virtual Machine (provisioned with Vagrant) for building Spark Notebook applications
author: paulovn
created: 2015-09-28 14:02:20
jupyter notebook python r spark virtual-machine jupyter notebook


sift : Knowledge extraction from web data
author: wikilinks
created: 2015-09-29 08:02:03
big-data entities knowledge-graph machine-learning natural-language-processing spark wikidata wikipedia python


graphx-citymap : CityMap coding test plus 3 solutions, 1 with Spark/GraphX
author: fancellu
created: 2015-10-06 09:57:38
graphx scalatest spark scala


ColumbiaImageSearch : Columbia Image and Face Search tool for MEMEX
author: ColumbiaDVMM
created: 2015-10-06 19:37:45
columbia-university face-api face-detection face-recognition hashing hbase image image-search kafka lmdb lopq memex python rest-api spark


Cookies help us deliver our services. By using our services, you agree to our use of cookies Learn more