Hadoop Administrator (CDH Operations Engineer) Poly
Hadoop/CDH operations engineer. Perform cluster and system performance tuning. Work with cluster subscribers to ensure efficient resource usage in the cluster and alleviate multi-tenancy concerns. Coordinate and monitor cluster upgrade needs. Monitor cluster health and build pro-active tools to look for anomalous behavior. Manage several Apache Blur search clusters. Monitor custom nightly batch workflows and ensure new data (over 150 billion records) are discoverable in Apache Blur.
1. Manages the functionality and efficiency of a group of computers running on one or more operating systems.
2. Maintains the integrity and security of servers and systems.
3. Sets up administrator and service accounts.
4. Maintains system documentation
5. Interacts with users and evaluates vendor products.
6. Makes recommendations to purchase hardware and software, coordinates installation and provides backup recovery.
7. Develops and monitors policies and standards for allocation related to the use of computing resources.
8. May program in an administrative language.
9. Oversees implementation of testing strategies and documentation of results.
10. Provides advice and training to end-users.
11. Provides guidance and work leadership to less-experienced staff members, and may have supervisory responsibilities.
12. Serves as a technical team or task leader.
13. Maintains current knowledge of relevant technologies as assigned.
14. Participates in special projects as required.
Bachelors Degree in Computer Science or a related technical discipline, or the equivalent combination of education, technical certifications or training, or work experience.
BASH shell fluency. Experience operating Hadoop 1 & 2 clusters, specifically Hive. CDH 4.x and 5.x familiarity. Linux system administration best practices, solid understanding of network services (DNS, LDAP). Experience with AWS services (VPC, EC2, Cloudformation) desirable. X.509 certificate management a plus.
10-15 years of directly related experience in systems administration and analysis.