bashreduce: span common Unix tools across multiple machines/cores

bashreduce : mapreduce in a bash script

bashreduce lets you apply your favorite unix tools in a mapreduce fashion across multiple machines/cores. There’s no installation, administration, or distributed filesystem. You’ll need:

• br somewhere handy in your path
• vanilla unix tools: sort, awk, ssh, netcat
• password-less ssh to each machine you plan to use

"MapReduce is a framework for computing certain kinds of distributable problems using a large number of computers (nodes), collectively referred to as a cluster. Computational processing can occur on data stored either in a filesystem (unstructured) or within a database (structured).

"Map" step: The master node takes the input, chops it up into smaller sub-problems, and distributes those to worker nodes. (A worker node may do this again in turn, leading to a multi-level tree structure.)

The worker node processes that smaller problem, and passes the answer back to its master node.

"Reduce" step: The master node then takes the answers to all the sub-problems and combines them in a way to get the output - the answer to the problem it was originally trying to solve.

The advantage of MapReduce is that it allows for distributed processing of the map and reduction operations. Provided each mapping operation is independent of the other, all maps can be performed in parallel - though in practice it is limited by the data source and/or the number of CPUs near that data. Similarly, a set of 'reducers' can perform the reduction phase - all that is required is that all outputs of the map operation which share the same key are presented to the same reducer, at the same time. While this process can often appear inefficient compared to algorithms that are more sequential, MapReduce can be applied to significantly larger datasets than that which "commodity" servers can handle - a large server farm can use MapReduce to sort a petabyte of data in only a few hours. The parallelism also offers some possibility of recovering from partial failure of servers or storage during the operation: if one mapper or reducer fails, the work can be rescheduled -assuming the input data are still available." - [Wikipedia]

For example:

br -m "host1 host2 host3"

• br < input > output
• br -r "uniq -c" < input > output
• LC_ALL='C' br -r "join - /tmp/join_data" < input > output

Very nice! Head on over to http://github.com/erikfrey/bashreduce/tree/master

LX260 (LG Rumor) Codes

##33284# (##DEBUG#) Debugging
##3282# (##DATA#) PCS Vision & Data
##2769737# (##BROWSER#) Browser Settings
##786# (##RTM#) Life Timers & Memory Reset
##477# (##GPS#) GPSOne Settings & Tests
##2342# (##CDG2#) CDG2 Menu (NAI setting here)
##8626337# (##VOCODER#) Vocoder Settings
##7738# (##PREV#) MOB_P_PREV (P_REV Prefs)
##2539# (##AKEY#) AKey Menu
##7764726# (##PROGRAM#) Requires Service Code

You are able to obtain your MSL code by creating an account at https://developer.sprint.com/ and registering your ESN/MEID number.  You must then call sprint and ask for the MSL code.  When they ask what you will be using it for, tell them you are a developer.

Cisco does Disco Commercial - I Will Survive

The human powered network is...

[RSS: visit this post for the video]

The NET Keyboard - Portable alternative to a NET Book or PC: Asus Wireless eee Keyboard

This is an impressive alternative to a Net Book or laptop for on the go computer access with tons of multimedia capabilities - the eee Keyboard PC from Asus.

This device has an 800x480 touch pad display interface attached to its aluminum keyboard.  It has HDMI & VGA out, multiple wireless data access capabilities, and comes with an easy to use, fun, and powerful control panel called the "eeefun control panel".

Tux is a Bath Fixer?

I took this picture several months ago on the way downtown.  I didn't know Tux, the Linux operating system mascot, fixed baths!?!

I am imagining the detail artist did not have a clue to the source of the decale.