Aim

I am a Node on the Edge because I cannot claim to have all the answers or a decade long history of experience but I can help to define questions to be solved. Hopefully their is a potential finite set of questions that allow most requirements be solved and my aim is define these with collaboration of the community.
It would be amazing if these questions developed into some kind of NP Complete space where they all referred to one question such in NP Complete Theory or Monty Pythons Holy Grail but that is/maybe asking too much?

Wednesday 2 September 2015

Node starting from the first Edge: Short History of Big Data from the start


              Catalysts                                                                                

information Overload                                                              tabulating Machine

 "The end of the quilt pen"                                                      "Automation of documentation"
                                                                                                 
e.g. The 1880 US Census took 8 year                                      Hollerith Tabulating Machine 1881 
to tabulate and it was estimated 1890                                      used to tabulate the census in one        
would take more than 10 years                                                year      
In 1970 A Toffler defines causes see more


documenting for knowledge research                                     expansion of libraries

  " When records began"                                                         "Accessible records"

e.g. The 1932 US population boom                                                      
and more documentation Social Security ID etc

expansion and requirement for communication                     The setting up of Networks

 " Instant communication "                                                    " Connected Devices"

e.g The invention of the telephone from the                        e.g. Radio's, LAN's, Mobile Telephones
1st patent by A G Bell to mass use from                                     and world wide web with IP
1930's to near universal usage from 1960's

defining of Information Theory                                          

 " Clear Signals "

e.g. Shannon and Weaver published in 1949
The Mathematical Theory of Communication
to define how to send and define information
to be measured or manipulated


Business Intelligence                                                                                        




                                                                       BIG DATA


Determined:



3D Data Management Controlling Data Volume Velocity Variety Doug Laney 2001


What is it? 

Volume - How much

Velocity - How produced at what speed

Variety - How much variety in attributes



Image result for covered in big data  Image result for covered in big dataImage result for elephant covered in big data



Data Source

Sensors on processes      Sensors on interactions of Processes        Sensors on interaction of Objects
                                                                                                               and processess


Data Storage

Technologies


Apache Hadoop


Data Processing

MapReduce






No comments:

Post a Comment