大数据工程师职位描述模板

Company Introduction

{{写一段简短而醒目的关于你公司的文字. Make sure to provide information about the company culture, perks, and benefits. Mention office hours, remote working possibilities, 以及所有你认为能让你的公司有趣的东西. Big Data Engineers like to work on huge problems - mentioning the scale (or the potential) can help gain the attention of top talent.}}

Job Description

我们正在寻找一个大数据工程师，将收集工作, storing, processing, 以及对海量数据的分析. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.

Responsibilities

• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
• Implementing ETL process {{如果从现有数据源导入数据是相关的}}
• Monitoring performance and advising any necessary infrastructure changes
• 定义数据保留策略
• {{添加任何其他相关职责}}

Skills and Qualifications

• 精通分布式计算原理
• 管理Hadoop集群，包括所有的服务 {{除非你将拥有特定的大数据开发运维角色}}
• 能够解决集群运行中出现的任何问题 {{除非你将拥有特定的大数据开发运维角色}}
• 熟练使用Hadoop v2, MapReduce, HDFS
• Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming {{如果流处理与角色相关}}
• 熟悉大数据查询工具，如Pig、Hive、Impala
• Experience with Spark {{如果您正在包含或计划包含它}}
• 具有集成多个数据源数据的经验
• 有使用NoSQL数据库的经验，如HBase, Cassandra, MongoDB
• 了解各种ETL技术和框架，如Flume
• 有各种消息传递系统的经验，如Kafka或RabbitMQ
• 有使用大数据ML工具包的经验，如Mahout, SparkML或H2O {{if you are going to integrate Machine Learning in your Big Data infrastructure}}
• Good understanding of Lambda Architecture, along with its advantages and drawbacks
• 有使用Cloudera/MapR/Hortonworks的经验 {{you can specify the distribution you are currently using or planning to use here}}
• {{列出您正在使用或计划使用的任何其他技术. 大多数大数据工程师都知道下面列出的一些: The Hadoop Ecosystem Table}}
• {{列出您需要的教育程度或证书}}