{"id":20,"date":"2009-03-05T20:09:33","date_gmt":"2009-03-06T01:09:33","guid":{"rendered":"http:\/\/jamesdevine.info\/?page_id=20"},"modified":"2015-01-14T12:40:15","modified_gmt":"2015-01-14T17:40:15","slug":"hadoop-cluster","status":"publish","type":"page","link":"https:\/\/jamesdevine.info\/index.php\/projects\/hadoop-cluster\/","title":{"rendered":"Hadoop Cluster"},"content":{"rendered":"

About<\/h3>\n

Hadoop is a Distributed File System written in Java that supports MapReduce. This project looked at the scalability of Hadoop MapReduce on a growing cluster size with a fixed problem. The study was run on both a real and virtual cluster.<\/p>\n

Results of a Scalability Performance Study<\/h3>\n

\"\"<\/p>\n

Hadoop Scalability Report<\/a><\/p>\n

Hadoop Scalability Slides<\/a><\/p>\n

Source Code<\/h3>\n

The source code consists of several bash scripts that were written to carry out the experimentation.<\/p>\n

\n

Source Code and Scripts<\/a><\/p>\n

Input<\/h3>\n

The input consists of a variety of free ebooks that were downloaded from Project Gutenberg.<\/p>\n

Input<\/a><\/p>\n

Output<\/h3>\n

The output of the experiments can be downloaded below.<\/p>\n

Output<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

About Hadoop is a Distributed File System written in Java that supports MapReduce. This project looked at the scalability of […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":5,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"class_list":["post-20","page","type-page","status-publish","hentry"],"aioseo_notices":[],"jetpack-related-posts":[{"id":5,"url":"https:\/\/jamesdevine.info\/index.php\/projects\/","url_meta":{"origin":20,"position":0},"title":"Archive","author":"James Devine","date":"March 5, 2009","format":false,"excerpt":"This page highlights some old projects I've worked on","rel":"","context":"In "General"","block_context":{"text":"General","link":"https:\/\/jamesdevine.info\/index.php\/category\/general-information\/"},"img":{"alt_text":"xen","src":"https:\/\/i0.wp.com\/jamesdevine.info\/wp-content\/uploads\/2009\/03\/xen.jpeg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":45,"url":"https:\/\/jamesdevine.info\/index.php\/projects\/virtualization-performance\/","url_meta":{"origin":20,"position":1},"title":"Virtualization Performance","author":"James Devine","date":"March 6, 2009","format":false,"excerpt":"During my first summer as an intern at MITRE I ran a comprehensive performance study on the VMware ESX cluster in the lab. I was able to determine the affects of the underlying storage system on performance. Using the results of disk I\/O and addition network I\/O tests I was\u2026","rel":"","context":"In "General"","block_context":{"text":"General","link":"https:\/\/jamesdevine.info\/index.php\/category\/general-information\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2,"url":"https:\/\/jamesdevine.info\/index.php\/about\/","url_meta":{"origin":20,"position":2},"title":"About","author":"James Devine","date":"March 5, 2009","format":false,"excerpt":"My name is James Devine and I'm a Principal Solutions Architect at Aviatrix. I have a deep passion for technology. I love diving in deep, learning everything I can, and then making that knowledge digestible for the masses - be it techies and c-suites alike. I've been fortunate enough to\u2026","rel":"","context":"In "General"","block_context":{"text":"General","link":"https:\/\/jamesdevine.info\/index.php\/category\/general-information\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3393,"url":"https:\/\/jamesdevine.info\/index.php\/pages\/about-james-devine\/","url_meta":{"origin":20,"position":3},"title":"About James","author":"James Devine","date":"December 11, 2014","format":false,"excerpt":"[vc_row unlock_row_content=\"yes\" row_height_percent=\"100\" override_padding=\"yes\" h_padding=\"0\" top_padding=\"0\" bottom_padding=\"0\" back_color=\"color-xsdn\" overlay_alpha=\"50\" equal_height=\"yes\" gutter_size=\"3\" column_width_percent=\"100\" shift_y=\"0\" z_index=\"0\" uncode_shortcode_id=\"115944\" back_color_type=\"uncode-palette\"][vc_column column_width_percent=\"100\" gutter_size=\"3\" expand_height=\"yes\" style=\"dark\" overlay_alpha=\"50\" shift_x=\"0\" shift_y=\"0\" z_index=\"0\" medium_width=\"0\" width=\"1\/1\"][vc_row_inner row_inner_height_percent=\"0\" overlay_alpha=\"50\" equal_height=\"yes\" gutter_size=\"0\" shift_y=\"0\" z_index=\"0\" limit_content=\"\" uncode_shortcode_id=\"342055\"][vc_column_inner column_width_use_pixel=\"yes\" gutter_size=\"1\" override_padding=\"yes\" column_padding=\"2\" expand_height=\"yes\" overlay_alpha=\"50\" shift_x=\"0\" shift_y=\"0\" shift_y_down=\"0\" z_index=\"0\" medium_width=\"0\" mobile_width=\"0\" width=\"1\/4\" mobile_height=\"300\" uncode_shortcode_id=\"126155\"][vc_single_image media=\"103360\"\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":186,"url":"https:\/\/jamesdevine.info\/index.php\/projects\/cuda-parallel-merge-sort\/","url_meta":{"origin":20,"position":4},"title":"CUDA Parallel Merge Sort","author":"James Devine","date":"May 3, 2009","format":false,"excerpt":"This work was part of a final project for Programming Languages. The CUDA API was investigated and used to write a parallel merge sort algorithm that executes on the Graphics Processing Unit(GPU), through a technique called GPGPU. GPGPU stand for General Purpose computing on a GPU. GPGPU allows for the\u2026","rel":"","context":"In "General"","block_context":{"text":"General","link":"https:\/\/jamesdevine.info\/index.php\/category\/general-information\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":29,"url":"https:\/\/jamesdevine.info\/index.php\/projects\/a-solution-to-the-traffic-jam-game\/","url_meta":{"origin":20,"position":5},"title":"A* Solution to the Traffic Jam Game","author":"James Devine","date":"March 6, 2009","format":false,"excerpt":"About A* The A* search algorithm is a search algorithm that uses a heuristic to estimate the cost of taking a given path in the solution tree to the goal state. The cost is calculated by adding the g + h values. The g value is the cost in steps\u2026","rel":"","context":"In "General"","block_context":{"text":"General","link":"https:\/\/jamesdevine.info\/index.php\/category\/general-information\/"},"img":{"alt_text":"traffic_jam","src":"https:\/\/i0.wp.com\/jamesdevine.info\/wp-content\/uploads\/2009\/03\/traffic_jam.jpg?resize=350%2C200","width":350,"height":200},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/pages\/20","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/comments?post=20"}],"version-history":[{"count":29,"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/pages\/20\/revisions"}],"predecessor-version":[{"id":412,"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/pages\/20\/revisions\/412"}],"up":[{"embeddable":true,"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/pages\/5"}],"wp:attachment":[{"href":"https:\/\/jamesdevine.info\/index.php\/wp-json\/wp\/v2\/media?parent=20"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}