[Tutorial] Deploying Kubernetes to AWS using KOPs

https://codefresh.io/blog/tutorial-deploying-kubernetes-to-aws-using-kops/

This tutorial will show you how to quickly and easily configure and deploy Kubernetes on AWS using a tool called kops. If you’re looking for a managed solution, we suggest using Stackpoint Cloud to do a one-click deployment of Kubernetes to AWS. We also suggest looking at our Kubernetes Cloud Hosting Comparison. Otherwise, let’s get started!

Step 1: Prepare your Host Environment

The first thing that you need to do is prepare your host environment. You’ll require a few pieces of software that let you create and manage your Kubernetes cluster in AWS.

Install Kubectl

Kubectl is the command line tool that enables you to execute commands against your Kubernetes cluster. Installation is as simple as downloading the binary, marking it as executable, and adding it to your path.

Mac (Homebrew)

Mac (manual)

Linux

Install kops

Kops is an official tool for managing production-grade Kubernetes clusters on AWS. It also supports other cloud providers as alpha features. A simple way to think about it is “kubectl for clusters” — these commands enable you to configure and build your cluster. You can download the binary directly from GitHub, or use homebrew if you are on a Mac.

Mac

Linux
First, download the latest binary from kops release page, then you’ll mark it as executable and put it on your path.

Install AWS CLI tools

The AWS CLI will let you directly interact with Amazon Web Services. The CLI is distributed as a Python package.

Install AWS CLI with Homebrew

Install AWS CLI with Pip

If you’re worried about version conflicts, consider installing the AWS CLI in a virtual environment.

Configure AWS CLI tools

After you’ve installed the AWS CLI, you need to configure it to work with your AWS account. This is easily done with the following command:

Step 2: Configure Route 53 Domains

Kops requires that your cluster use DNS. Using DNS names is much easier than remembering a set of IP addresses, and it is easier to scale your cluster as well. If you are going to eventually run multiple clusters, each cluster should have its own subdomain as well. As an example:

In our example, we’ll use Route 53 Hosted Zones to configure our cluster to use dev.yourdomain.com . From there, we’ll be able to assign subdomains to our Kubernetes clusters. You can either use AWS’s Hosted Zones documentation to configure it or simply use the following command:

Now that you’ve set up your Hosted Zone, you need to configure the NS records for the parent domain to work with your subdomain. In this scenario, yourdomain.com  is the parent domain and dev  is the subdomain. You’ll need to create an NS record for dev  in yourdomain.com . This ensures that dev.yourdomain.com  correctly resolves.

If your domain is hosted in Route 53 already, create a new record NS record, and copy and paste the name servers from the subdomain. This will allow requests to reference the hosted zone we just created.

Every hosting provider exposes this functionality, but it can be a bit complicated. To validate that you’ve done it correctly, try running dig NS dev.yourdomain.com  If it responds with 4 NS records that point to your Hosted Zone, then everything is working properly.

Step 3: Create S3 buckets for storage

The next step is to configure S3 buckets to store your Kubernetes cluster configuration. Kops will use this backend storage to persist the cluster configuration. Setting up S3 buckets is very easy. You can create it online, or use the CLI. We recommend using the CLI. Since we went with dev.yourdomain.com , let’s name our S3 bucket storage.dev.yourdomain.com .

After you run this, you’ll need to export the environment variable so that kops uses the right storage.

Optionally, you can export the variable in your bash_profile  file so you don’t have to worry about configuring it every time you need to use kops.

Step 4: Build Cluster Configuration

Now that you’ve got S3 storage configured, it’s time to build your Kubernetes cluster configuration. Execute the following command:

After this completes, your cluster’s configuration will be built, and kops will output a few interesting commands to manage your cluster. Note that your cluster hasn’t actually been built, just the configuration.

Step 5: Build your Kubernetes Cluster

Now that your cluster’s configuration is built, execute the following command to instruct kops to actually build the cluster in AWS

After a few minutes, your Kubernetes cluster will be deployed in AWS. If you make any additional configuration changes, just use kops update cluster again to push your changes.

Step 6: Add Cluster to Codefresh

Now we have a cluster up. Let’s use it! First, configure  kubectl  to work with the cluster we just made.

Now all kubectl  commands will automatically point at the cluster we just created.

Adding to Codefresh is very easy, after logging in/creating a free account, click on “Kubernetes” on the left-hand side and then “Add Cluster”.

“Click Add Provider” and select “Amazon AWS”. Expand the provider dropdown to now connect your cluster. Name it whatever you like and then to get connection information using kubectl . I’ll be using pbcopy to copy each command to the clipboard, but if you don’t have that installed you can remove that part of the command.

The output of each command will be copied to your clipboard so you can paste it into Codefresh.

Host

Certificate

Token

Click “Test” to make sure the connection is working then go back to the Kubernetes page. Click “Add Service” and add the information below

Click “Deploy” and nginx will be up and running within a few seconds.

Since we exposed it publicly, you’ll see a new ELB setup in AWS for the service we just deployed.

Conclusion

While Kops isn’t as complete a solution as a fully-managed Kubernetes service like GKE, it’s still awesome. The setup time and flexibility make Kops + AWS a great way to get up and running with a production-grade cluster. The next step is to add a real application to Codefresh. Then you can build, test, and deploy the application from a single interface. Plus we can shift all the testing left, and spin up environments on-demand for every change.

Learn more about Codefresh.

Fix Window ACL permission issue

To reset files permissions, follow these easy 3 steps:

Step 1: Run “cmd” as Administrator.

Note: In MS Windows 8 and above, press the Win+X keys, then choose “Command Prompt (Admin)”.

Step 2: Go to the drive or folder in question, for example:

CD /D D:

Step 3: To reset all the files permissions, type:

icacls * /T /Q /C /RESET

Fix ownership:

takeown -r -f d:\sharename

 

Licensing How To: When do I need a Client Access License (CAL)

https://blogs.technet.microsoft.com/volume-licensing/2014/03/10/licensing-how-to-when-do-i-need-a-client-access-license-cal/

SUMMARY:  Knowing who, when, and what needs a Client Access License (CAL) is a great question and one our team answers often. Under most scenarios, CAL requirements are generally straight forward, however, there are several specific scenarios which we address below.   In this Licensing How To post, we cover the basics of Client Access Licensing, and recap a few common scenarios which may apply to you.

The Licensing How To series posts are provided by our Customer Service Presales and Licensing team members.  These scenario based licensing topics are written on trending topics and issues based on their interactions with customers, Partners, and field sellers.  For more posts from the Licensing How To series, search the “Licensing How To” tag on this blog.

 

It’s a question we answer daily, “I have scenario X, Y, or Z. Do I need a CAL?”  Server software licensed via the Server / CAL licensing model always requires some sort of server license (which may be per instance or per processor depending on the Product) as well Client Access Licenses (CALs) for users and/or devices to access the server software.  However, the question of who or what needs a CAL, along with any noted exceptions, varies by product.

The general requirement is, any User or Device that accesses the server software, either directly or indirectly, requires a CAL.  Depending on the product and functionality being accessed, additive CALs may be required as well.

External users* (users who are not employees, onsite contractors, or onsite agents) can be licensed with CALs, External Connector licenses, and in some cases (SharePoint 2013, Lync 2013 or Exchange 2013) – external user access is granted with the Server License.

Access requirements vary by Product and you need to evaluate the requirements for each product you use. We encourage you to review the Product Use Rights, or Microsoft Software License Terms, that are applicable to you, and the products you use.

Here are a CAL questions that we answer frequently and we thought sharing them might help you when you think about your own CAL requirements. Please note that the below scenarios are based on licensing for the Server / CAL products currently available as of the date of publishing of this post.


Top CAL Questions


1 – Does my Multifunction Printer need a CAL?

Yes, if the multifunction printer is connected to a Windows Server network.  A multifunction printer accesses server software to; receive an IP address, to receive a job, to communicate that the job is finished, etc.  In short, it communicates with the server software.  If the multifunction printer is accessing any server software licensed via the Server / CAL licensing model it requires a CAL for that software. The one caveat is, if your users who use the printer have CALs then the printer is covered by their use via their CALs. If not then the printer itself requires a device CAL. The same CAL requirement applies to any other type of networked device – such as networked scanners, networked fax machines, etc.  Devices that do not connect to the network or the server software (generally referred to as peripherals) do not require CALs.


2 – Do my servers need a CAL?

Generally speaking – server to server communication does not require a CAL.  However, servers used to pool connections (sometimes referred to as multiplexing) does not reduce your CAL licensing requirements.  If, for example, you have an application server which uses SQL Server for its database – users of the application (or the devices they use) will need a SQL CAL even though they may not access the SQL Server directly.  If you use a Linux server to run a web server, but your users accessing the web server are being authenticated via Windows Server – users (or the devices they use) will need a Windows Server CAL.


3 – Do my external users need a CAL?

The general rule is all server software access requires a CAL.  However, external users* may have additional licensing options depending on the product.  For example, with Windows Server – external users can be licensed with CALs or External Connectors (whichever is more cost effective).  External user access to application servers such as SharePoint 2013, Lync 2013, and Exchange 2013 is included with the server software – CALs or External Connectors are not required for external users for these products.  Note: external users will need to be licensed appropriately for the underlying Windows Server operating system and related software such as SQL.


4 – Do I need an RDS CAL?

There are two basic scenarios which trigger the requirement for an RDS CAL.

    1. Your users or devices directly or indirectly access any of the RDS product functionality, and/or
    2. Your users of devices directly or indirectly interact with the graphical user interface of the server software using RDS functionality or other third party technology (e.g. Citrix, GraphOn, 2X to name a few)

If you meet either (or both) of the points described above – an RDS CAL is required.  It is also worth pointing out that RDS CALs are required in a VDI deployment when any of the RDS components are used to support it (e.g. Remote Desktop Web Access, Remote Desktop Gateway, Remote Desktop Connection Broker, Remote Desktop Session Host, or the Remote Desktop Virtualization Host.


5 – Do I need a CAL when my Windows Server is used to run a web server?

Windows Server 2012 R2 configured to run Web Workloads ** do not require CALs or External Connectors.  Web workloads, also referred to as an internet web solution, are publically accessible (e.g. accessible outside of the firewall) and consist only of web pages, web sites, web applications, web services, and/or POP3 mail serving.  Access to content, information, and/or applications within the internet web solution must be publically accessible.  In other words, they cannot be restricted to you or your affiliate’s employees.

If you have Windows Servers configured to run a “web workload” these users will not require CALs or External Connectors.  However, let’s say you are using Windows Server to setup an online store where customers can buy widgets.  You have front end Windows Servers setup to support your website, and backend servers (e.g. commerce servers) setup so customers can check out and buy your widgets.  The front end servers used to host your website would generally be considered as running “web workloads” and CALs or External Connectors will not be required to access these servers.  Once the customer adds a widget to their shopping cart, creates an account and enters their credit card and shipping information to complete the sale – they are now authenticated via your back end commerce servers/application (non-web workload).  Since users are accessing the backend commerce servers which web workloads are not running – CALs or External Connectors will be required for users to access these back end servers.


6 – Can I use my CALs to access someone else’s server?

You may use CALs purchased by your company to access your servers, or servers owned by your Affiliates*** only.  You may not use your CALs to access servers owned by an un-affiliated third party.  Let’s say for example, that Company A and Company B are affiliates.  Company A wants to provide employees from Company B with access to their SharePoint Servers.  However, Company B already owns Windows Server and SharePoint Server CALs (that match the version of Windows and SharePoint Server that Company A uses).  Company A will not need to purchase additional Windows Server or SharePoint Server CALs since employees from their affiliate, Company B, are already covered with the appropriate CALs.  If we use the same scenario as above, but assume Company A and B are not affiliated – then CALs owned by B cannot be used to access Company A’s servers.  Company A would need to appropriately license their SharePoint farm for external users.


7 – Do I need CALs for my administrators?

Server software licensed using CALs permits up to 2 users or devices to access the server software for the purposes of administration without CALs.  However, if your administrators also use the software for anything other than administration (for example, they check their email), CALs will be required for them as well.

For additional information on CAL requirements, consult your Product Use RightsLicense Terms, or contact your Reseller, Microsoft Partner, or Account Team.  Many products have licensing guides on the Volume License Website and/or their respective product sites.  Here are a few of our favorite resources on the subject.

Multiplexing – Client Access License (CAL) Requirements
Base and Additive Client Access Licenses: An Explanation
Licensing Windows Server 2012 R2 Remote Desktop Services
About Licensing – Client Access Licenses and Management Licenses

 

External Users means users that are not either your or your affiliates’ employees, or your or your affiliates’ onsite contractors or onsite agents.

** Web Workloads (also referred to as “Internet Web Solutions”) are publicly accessible and consist solely of web pages, websites, web applications, web services, and/or POP3 mail serving. For clarity, access to content, information, and applications served by the software within an Internet Web Solution is not limited to your or your affiliates’ employees.

Software in Internet Web Solutions is used to run:

  • web server software (for example, Microsoft Internet Information Services), and management or security agents (for example, the System Center Operations Manager agent).
  • database engine software (for example, Microsoft SQL Server) solely to support Internet Web Solutions.
  • the Domain Name System (DNS) service to provide resolution of Internet names to IP addresses as long as that is not the sole function of that instance of the software.

*** “Affiliate” means any legal entity that a party owns, that owns a party, or that is under common ownership with a party.  “Ownership” means, for purposes of this definition, control of more than a 50% interest in an entity.

This is one scenario and licensing situation. Each customer scenario can vary by deployment, usage, product version, and product use rights.  Always check your contract, and the current Products Use Rights document to confirm how your environment should be fully licensed.  The blogging team does not warrant that this scenario will be the right licensing solution for other similar cases.

 

Additional content in response to questions:

Following is our best effort to answer some of the questions posted below.  Please know that your feedback is shared with the respective product groups within Microsoft.  The purpose of the Licensing How To series is to share insights gained through the many interactions our customer service teams have with customers, and present the details based on the licensing requirements set forth in our license term documents and guidance from the respective product groups.  We are listening and take your feedback to heart.

Before going into further details, lets include the actual language from the use terms regarding CALs.  This is from the current Product Use Rights for volume license customers, but the license terms for OEM and Retail licenses will be similar if not identical.  Because the majority of questions are around Windows Server CAL requirements – this is from the Windows Server license terms.

  1. You must assign each CAL to a user or device, as appropriate, and each External Connector License to a Licensed Server.
  2. CALs or External Connector Licenses are required for access to server software.
  3. CALs and External Connector Licenses permit access to the corresponding version (including earlier versions used under downgrade rights) or earlier versions of server software.
  4. CALs are not required for access by another Licensed Server or for up to 2 users or devices to administer the software.
  5. CALs are not required to access server software running a Web or HPC Workload.
  6. CALs not required for access in a Physical OSE used solely for hosting and managing Virtual OSEs.
  7. Your CALs and External Connector Licenses only permit access to your Licensed Servers (not a third party’s).

Q1 – If I have a printer that uses an IP address assigned by a router, but the drivers are deployed via a GPO…does that need a CAL?

A1 – Yes, any Windows Server access requires a Windows Server CAL.  In this scenario, the printers are connecting to, and receiving benefit of, Windows Server.  However, if all users who access or use that printer already have a user CAL – then you’re covered and will not need additional device CALs for the printer.

 

Q2 – If I have guests that come into my office an temporarily use a Windows DHCP server to grab an IP address to access the Internet, do they need CALs? I guess the takeaway is to never use a Windows DHCP server?

A2 – Yes, they are using a Windows Server service and would need a CAL.

 

Q3 – If I have a Remote Access card installed in a server, does that need a CAL? If I run sniffer, does that need a CAL? If I use a common management tool that installs a service/daemon on each server – does that need a CAL?

A3 – Peripherals, server components and network equipment on their own do not generally require a CAL (for Windows Server or otherwise).  Server to server communication does not require a Windows Server CAL (between two licensed Windows Servers).  Device CALs are intended for the clients/endpoints accessing the Windows Server (for any reason – to get an IP address, to access a file, to authenticate to AD, to access an application of any type on the Windows Server, etc.)  User CALs are intended for the same reason – but are assigned to the users using the clients/endpoints.  For example, a sniffer.  Generally, these won’t require a CAL – they simply monitor network traffic.  However, let’s say that you have a software based sniffer installed on your desktop at work – and your desktop is accessing Windows Server (to get an IP, to authenticate to AD, etc.)  This scenario will require a device CAL for your desktop (or a user CAL for you), not because you are using a sniffer, but because the device/endpoint it’s installed on accesses Windows Server.  Management software.  Let’s use the same concepts above.  Let’s say for example, the management software is installed on a Windows Server, and is being used to manage client devices, network equipment, and other servers.  Any device that accesses Windows Server as a result of being managed will require a Windows Server device CAL (with the exception of other Windows Server since Windows Server to Windows Server communication does not require a CAL).  If you are licensed by user in this scenario however, and all users are covered with a Windows Server CAL, the n you’re covered since all users of the managed devices are already covered with user CALs.

 

Q4 – If I manage another companies servers as their help desk…and employ more than 3 people…and I already have CALs for my people, do I need additional CALs to administer their network? 

A4 – CALs (for Windows Server, Windows Server RDS, Exchange Server, SharePoint, SQL, Lync, etc.) can be used only to access organization owned servers.  Your CALs cannot be used to access servers operated by an independent organization (see #6 in the blog above).  That being said, their licensed servers will provide the ability for 2 users/devices to access the servers for purposes of administration.  For any number of users/devices employed to manage their servers in excess of 2, additional CALs will be required.  For example, if 10 people, using 10 different devices are employed to manage their servers – a total of 8 additional user or device CALs will be required.  The organization whose servers are being accessed will be required to purchase these CALs however.  It is worth pointing out, that certain products – such as Exchange Server 2013, SharePoint Server 2013, and Lync Server 2013 (as well as to some extent, CRM Server 2013) do not require CALs for external user (non-employee) access (administrative access or otherwise).  In this example, you are being hired by a third party to provide help desk support OFFSITE and would be considered an external user (see #3 in the blog above).  If for example, you have 10 users/devices employed to administer a third parties servers – the company may be required to purchase additional CALs for products such as Windows Server, and/or (unless licensed by processor or core).  However, because you can leverage the external user CAL exception for Exchange Server 2013, Lync 2013, or SharePoint 2013 – additional CALs for these products would not be required in this scenario.  Note the external user CAL exception for the products mentioned above apply only to the current versions and not prior versions. External users must be offsite.

 

Q5 – So, if I host a web server using a Web Workload for any reason I need CALs for each user?

A5 – CALs are not required to access servers running a web workload (defined above in the blog), so no – CALs are not required to access servers running a web workload when users access anonymously.

 

Q6 – If I assist another company in an e-mail migration to O365 and need administrative/test mailboxes – do those need CALs?

A6 – It’s best to look at each involved product individually.  For example, in this scenario you could potentially have Windows Server, Exchange Server, and Office 365 (or Exchange Online) – each with their own unique CAL licensing requirements.  For Windows Server, see A4 above as this would be considered administration.  For Exchange Server, we license by user or device accessing the Exchange Server (not per mailbox).  Exchange Server also has the same 2 user/device administration exception as described above.  If you have more than 2 users/devices accessing Exchange Server for the migration – yes additional CALs will be required.

  • Admin and test mailboxes just look like regular mailboxes from an O365 perspective, so they require subscriptions to be assigned.   (USL required)
  • Admins can administer Office 365 (including Exchange) without having a mailbox, so that answer is a little more nuanced than the test accounts one.
  • Regarding 30 day grace period, there is a very narrow scenario where this is enabled (when you have a hybrid deployment and you are moving mailboxes from on-premises) but it is not worth bringing up in this context.
  • Note that we offer free 30 day trials of 25 licenses per tenant for Office 365, so that’s how most customers accomplish their testing.

 

Q7 – When accessing a VDI, do I always need an RDS CAL even if I use a pure Citrix solution?

A7 – Note that we have amended the last sentence to number 5.  RDS CALs are required only when accessing the Windows Server GUI, or if any of the RDS components listed are used (e.g. Remote Desktop Web Access, Remote Desktop Gateway, Remote Desktop Connection Broker, Remote Desktop Session Host, or the Remote Desktop Virtualization Host.)  If you are using a third party VDI solution, and it does not use any of the RDS components, then RDS CALs are not required.

 

We apologize if we were unable to get to each and every question.  Please note this post was intended to be a general guide only and address some outlier scenarios we hear about from time to time.  We encourage everyone to leverage your resellers, partners, and/or distributors.  In each case, these resources have access to teams directly at Microsoft who can answer licensing questions, who are better equipped to answer large volumes of questions.  There is also a group available who handles licensing questions via the Sales and Partner Information line at800-426-9400 .

Again, we appreciate the feedback and please know that it is shared with our products teams.

Learn How to Install Ansible and Run Playbooks

https://www.linode.com/docs/applications/configuration-management/learn-how-to-install-ansible-and-run-playbooks

https://github.com/ansible/ansible-examples

Getting started with Ansible: An Installation and Configuration Tutorial

Consider the monotony of administering a server fleet; keeping them all updated, pushing changes out to them, copying files, etc. Things can get complicated and time consuming very quickly, but it doesn’t have to be that way.

Ansible is a helpful tool that allows you to create groups of machines, describe how those machines should be configured or what actions should be taken on them, and issue all of these commands from a central location. It uses SSH, so nothing needs to be installed on the machines you are targeting. Ansible only runs on your main control machine, which can even be your laptop! It is a simple solution to a complicated problem.

This guide will introduce you to the basics of Ansible. By the end of this guide, you’ll have the tools needed to turn a brand new Linode into a simple web server (Apache, MySQL, PHP), easily replicable and adjustable.

This guide is written for a non-root user. Commands that require elevated privileges are prefixed with sudo. If you’re not familiar with the sudo command, you can check our Users and Groups guide. Some systems may require you to run Ansible commands as root. If so, prefix the ansible commands in this guide with sudo.

Installing Ansible

Ansible only needs to be installed on the control machine, or the machine from which you will be running commands. This will likely be your laptop or other computer from which you frequently access your server, or it may be a centralized server in more complicated setups.

Make sure that you have Python 2.x available on the control machine. Ansible is not compatible with Python 3, nor can you use Windows as the control machine. You can build Ansible from source, or install the latest stable packages using the proper command below.

  • Mac OS X:
    1
    2
    sudo easy_install pip
    sudo pip install ansible
    
  • CentOS/Fedora:
    1
    sudo yum install ansible
    

    The EPEL-Release repository may need to be added on certain versions of CentOS, RHEL, and Scientific Linux.

  • Ubuntu:
    1
    2
    3
    4
    sudo apt-get install software-properties-common
    sudo apt-add-repository ppa:ansible/ansible
    sudo apt-get update
    sudo apt-get install ansible
    

First Steps to Configuring Ansible

Now that you have Ansible installed, let’s test it out on a known server. All Ansible commands are executed via the command line (or batched in scripts), and follow the pattern:

1
ansible server_or_group -m module_name -a arguments

We’ll get to groups in just a moment, but for now, let’s try to make a simple connection to an existing server you have using the ping module. In place of server_or_group, type the name of a server that you can currently SSH into from your machine using key authentication. If you are using a non-standard SSH port, include that after a colon on the same line (myserver.com:2222).

By default Ansible will use the same username as your current machine’s username. If this will not match up, pass the proper username in using the -u username argument.

If you don’t want to use SSH keys, you can add the --ask-pass switch.

To try an Ansible command without any additional setup, we’ll add a few extra arguments for now. Format a test command like the following:

1
ansible all -i myserver.com, -m ping

The extra directives are the all -i and the comma after your server name. This is temporary, and is only there to tell Ansible to try connecting directly to the server without an inventory file, which we’ll learn about later.

If you are successful you should see output similar to the following:

1
2
3
4
myserver.com | success >> {
    "changed": false,
    "ping": "pong"
}

You were just able to get a valid connection to your server via Ansible!

Using Ansible’s Inventory File

You executed an Ansible command against one client, but it would be cumbersome to have to type the host’s address every single time, and what if you had several servers you wanted to apply the same configuration to? This is where Ansible’s inventory file comes into play.

  1. By default, the inventory file is expected to be /etc/ansible/hosts. Create that path and file if it does not already exist.

    If you are running OS X, you may want to create your own Ansible directory elsewhere and then set the path in an Ansible configuration file:

    1
    2
    3
    mkdir ~/Path/To/ansible
    touch ~/Path/To/ansible/hosts
    touch ~/.ansible.cfg
    

    Open ~/.ansible.cfg file and add the following lines:

    ~/.ansible.cfg
    1
    2
    [defaults] 
    inventory = ~/Path/To/ansible/hosts 
    
  2. Add an entry to your hosts file, pointing to a server that you connected to in the previous section. You can include multiple servers in this file, using either domains or IP addresses, and can even group them:
    ~/Path/To/ansible/hosts
    1
    2
    3
    4
    5
    6
    mainserver.com
    myserver.net:2222
    
    [mailservers]
    mail1.mainserver.com
    mail2.mainserver.com 
    
  3. Use the all directive to ping all servers in your hosts file via Ansible:
    1
    ansible all -m ping
    

You should receive the same output as before, for each server in your hosts file. Note that instead of all, you could have substituted mailservers from the example file, and it would run just against those servers.

You can heavily customize the Inventory file, so check out the docs for it if you’re interested.

Ansible Configuration via Playbooks

Playbooks in Ansible define a series of actions to run, and address particular sets of servers. It’s important to note that, unlike some other configuration tools, a playbook does not describe a state of the machine, with Ansible determining all the changes that need to be made on its own. However, playbooks should be designed to be idempotent, meaning that they can be run more than once without negative effects. For example, a playbook might have a task that sets up a configuration file for a server and injects a few variables. The playbook should be written such that Ansible can take the template configuration file, compare it to the actual file, and create/update it only if necessary. Luckily, many Ansible modules take care of the heavy lifting for that.

You can write playbooks to perform initial server configurations, add users and directories, ensure certain software packages are installed or uninstalled, move files, etc. A playbook can also run a few commands on one set of machines, switch to a different set to run different commands, and then switch back to the original or a different set of machines. It is procedural, and tasks are run in order, top to bottom.

A playbook is a YAML file, and typically follows this structure:

Sample Playbook YAML file
1
2
3
4
5
6
---
- hosts: [target hosts]
  remote_user: [yourname]
  tasks:
    - [task 1]
    - [task 2] 

For example, the following playbook would log in to all servers in the marketingservers group and ensure Apache was started.

Sample service check playbook
1
2
3
4
5
6
7
8
---
- hosts: [marketingservers]
  remote_user: webadmin
  tasks:
    - name: Ensure the Apache daemon has started
      service: name=httpd state=started
      become: yes
      become_method: sudo 

In the playbook above is an example of a task:

Playbook task
1
2
3
4
5
  tasks:
    - name: Ensure the Apache daemon has started
      service: name=httpd state=started
      become: yes
      become_method: sudo

Every task should have a name, which is logged and can help you track progress. Following the name line is the module that will be run (in this case, the service module), and the other attributes provide more options, in this case instructing Ansible to use sudo privileges (which we will configure later).

Running Playbooks

Executing a playbook is even easier than running ad-hoc commands like we did earlier. Assuming you are in the same directory as a playbook file, you run the following command:

1
ansible-playbook myplaybook.yml

If you want to see what hosts this playbook will affect without having to open up the YAML file, you can run:

1
ansible-playbook myplaybook.yml --list-hosts

Types of Tasks You Can Run

Ansible ships with a large collection of modules that you can run as tasks or via ad-hoc commands. To see a listing of all available modules, run:

1
ansible-doc -l

A few common core modules you might be interested in learning first include:

Basic Web Server Setup via Ansible Playbooks

As an example, we’ll use Ansible to turn a freshly created Linode server into a web server, configured with Apache, MySQL, and PHP, ready to serve up dynamic sites and configured with the proper users and permissions. For brevity we won’t handle all of the features and configuration that might normally be involved, but will cover enough to get you started.

The following playbooks are for learning purposes only, and will NOT result in a hardened or secure server. Use them to learn from, but do not use them for production instances!

Prerequisites

  • This example will assume a brand new Ubuntu 14.04 LTS server, without any additional configuration already done to the box. The very first order of business will be to add in our public encryption keys so that we can connect without supplying passwords.
  • Because Ansible playbooks are idempotent and can be run repeatedly without error, the usertask checks that a user exists and that the password on file (which the system stores hashed) matches the hash you are supplying. Therefore you cannot (and should not) just put in a plaintext password, you must pre-hash it.
  • Create a password hash for Ansible to use when communicating with the servers. An easy method is to use Python’s PassLib library, which can be installed with sudo pip install passlib.

    Once installed, run the following command, replacing plaintextpassword with your actual password:

    1
    python -c "from passlib.hash import sha512_crypt; print sha512_crypt.encrypt('plaintextpassword')"
    

Create the System User

  1. Add your new server’s IP address to your Ansible hosts file so that we can address it. Remove any previous entries you may have added in the test sections above. Give the new server a group name to make it easier to refer to later. In our example the group name is linode.
    /etc/ansible/hosts
    1
    2
    [linode]
    123.123.123.123 
    
  2. Write a playbook that creates a new normal user, adds in our public key, and adds the new user to the sudoers file.

    We’re introducing a new aspect of Ansible here: variables. Note the vars: entry and the NORMAL_USER_NAME line. You’ll notice that it is reused twice in the file so that we only have to change it once. Replace yourusername with your choosen username, localusername in the path for the authorized_key, and the password hash.

    initialize_basic_user.yml
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    ---
    - hosts: linode
      remote_user: root
      vars:
        NORMAL_USER_NAME: 'yourusername'
      tasks:
        - name: "Create a secondary, non-root user"
          user: name={{ NORMAL_USER_NAME }} 
                password='$6$rounds=656000$W.dSlhtSxE2HdSc1$4WbCFM6zQV1hTQYTCqmcddnKrSXIZ9LfWRAjJBervBFG.rH953lTa7rMeZNrN65zPzEONntMtYt9Bw74PvAei0' 
                shell=/bin/bash
        - name: Add remote authorized key to allow future passwordless logins
          authorized_key: user={{ NORMAL_USER_NAME }} key="{{ lookup('file', '/Users/localusername/.ssh/id_rsa.pub') }}"
        - name: Add normal user to sudoers
          lineinfile: dest=/etc/sudoers
                      regexp="{{ NORMAL_USER_NAME }} ALL"
                      line="{{ NORMAL_USER_NAME }} ALL=(ALL) ALL"
                      state=present 
    
  3. Save the playbook file as initialize_basic_user.yml and run the playbook with the following command. Note how we specify the use of a particular user (-u root) and force Ansible to prompt us for the password (-ask-pass) since we don’t have key authentication set up yet:
    1
    ansible-playbook --ask-pass -u root initialize_basic_user.yml
    

You should see output from Ansible that reports that the three tasks all completed successfully with a status of “changed.” We can now work with new playbooks using our normal user account and keys.

Configure the Base System

Let’s take care of some common server setup tasks, such as setting the timezone, updating the hosts file, and updating packages. Here’s a playbook covering those steps:

common_server_setup.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
---
- hosts: linode
  remote_user: yourusername
  become: yes
  become_method: sudo
  vars: 
    LOCAL_HOSTNAME: 'web01'
    LOCAL_FQDN_NAME: 'www.example.com'
  tasks:
    - name: Set the timezone for the server to be UTC
      command: ln -sf /usr/share/zoneinfo/UTC /etc/localtime
    - name: Set up a unique hostname
      hostname: name={{ LOCAL_HOSTNAME }}
    - name: Add the server's domain to the hosts file
      lineinfile: dest=/etc/hosts 
                  regexp='.*{{ item }}$' 
                  line="{{ hostvars[item].ansible_default_ipv4.address }} {{ LOCAL_FQDN_NAME }} {{ LOCAL_HOSTNAME }}" 
                  state=present
      when: hostvars[item].ansible_default_ipv4.address is defined
      with_items: "{{ groups['linode'] }}"
    - name: Update packages
      apt: update_cache=yes upgrade=dist 

Run this playbook:

1
ansible-playbook common_server_setup.yml --ask-become-pass

As you run this playbook you will again see the steps come across as “changed.” Updating packages may take a few minutes, so don’t fret if it doesn’t return straight away.

Install the Stack

Finally, let’s get a very basic server set up with Apache and PHP, and a test MySQL database to use.

  1. The following playbook downloads the appropriate packages, turns on the Apache and MySQL services, and creates a basic database and user.
    setup_webserver.yml
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    ---
    - hosts: linode
      remote_user: yourusername
      become: yes
      become_method: sudo
      tasks:
        - name: "Install Apache, MySQL, and PHP5"
          apt: name={{ item }} state=present
          with_items:
            - apache2
            - mysql-server
            - python-mysqldb
            - php5
            - php-pear
            - php5-mysql
    
        - name: "Turn on Apache and MySQL and set them to run on boot"
          service: name={{ item }} state=started enabled=yes
          with_items:
            - apache2
            - mysql
    
        - name: Create a test database
          mysql_db: name=testDb
                    state=present
    
        - name: Create a new user for connections
          mysql_user: name=webapp 
                      password=mypassword 
                      priv=*.*:ALL state=present 
    
  2. Run the playbook from your control machine with the following command:
    1
    ansible-playbook setup_webserver.yml --ask-become-pass
    

    When this playbook finishes, visit your Linode’s IP address or FQDN to see the default Ubuntu Apache index page.

  3. Log in via SSH and check to see that the testDb has indeed been created:
    1
    2
     mysql -u root -p
     show databases;
    

    You can even create a sample PHP page and place it in /var/www/html to test that PHP is active on the server. Ansible has done as we instructed it to, installing the appropriate packages and setting things up as we want.

Exploring Ansible Further

This is just the start of learning Ansible, and as you continue to learn and explore you will find it a truly powerful and flexible tool. Take a look at some of the example Ansible playbooks provided by the company itself.

Below are a few topics to explore that become important as you create playbooks of any complexity, and that you will see frequently in others’ playbooks.

More Information

You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.