27 February, 2019 by macklus on Advanced concepts

Tips for migrating services on Amazon AWS

When working with large infrastructures, especially when we define a new infrastructure for an existing service, it is time to carry out the final change, and start using all services with the new environment.

If your new infrastructure is in Amazon AWS, these simple tips can make your migration more comfortable.

Use the weights of Route 53

The Route 53 DNS service allows you to assign weights to the different entries. This is very useful to be able to change the traffic gradually from the old IPs to the new ones, and see how our platform behaves as the traffic increases.

All we have to do is create two equal entries, each pointing to the new destination, and distribute the total weight (255) between each entry. As we change it, Amazon will direct more traffic to the new destination until the process is complete.

“Warm up” ELB balancers

Amazon’s infrastructure is reactive to the different increases or decreases in traffic that are occurring.

In ELB load balancers, when the traffic increases suddenly, it is normal that the IPs of these balancers change, and the latency decreases even if the traffic increases.

This seems to indicate that Amazon detects the load increase and switches to an ELB with more capacity that can guarantee an adequate response for the new traffic.

Therefore, before migrating, it is advisable to launch a high number of requests to the balancer (or to the entire infrastructure) using Jmeter or a similar program, so that the balancer detects the change of traffic and assigns us new balancers of more capacity. This way we will be more prepared to process the new traffic.

Load the cache

If your new infrastructure has a distributed cache system such as CloudFront or Fastly, we can anticipate the change and improve the performance of it (while reducing the load of the platform) by preloading content into it.

If we expect that most of the traffic comes from a specific area, when launching requests from that area the system will request and store the result in its cache, so when making the change it will not be necessary to make the requests again .

Leave static content generated

If our infrastructure is ready to generate static content (for example clippings with Lambda against S3), leaving already generated those contents that we hope will be used will avoid an unnecessary burden in the first moments of the change of platform.

Forecast scaling

Unless our platform is completely serverless , it is normal to expect that the EC2 instances will escalate (new instances will be activated due to an increase in load).

This process (depending on your configuration), can take several minutes, during which the performance of the platform can be reduced, or even stopped if the load increase is very high.

Therefore, for platform migration it is advisable to start with one or two more instances, which allow us to avoid problems if we receive more load than expected, or if we have dimensioned our resources by default.

In short, prepare for the worst and try to leave everything generated or prepared before you start, so that the change is as traumatic as possible.

16 February, 2019 by macklus on Marketplace

Register on AWS Marketplace

For all users of Amazon Web Services, Amazon has an online market where you can buy everything related to AWS. Thus, you can find AMI’s that come prepared with the specific software you need, CloudFormation scripts to create a specific platform, custom ECS containers, etc.

In addition, Amazon gives you the option to upload your products to your market and make money with them.

Although the process to register can be delayed in time (due to the different verifications required), it is not really complicated.

1.- Creation of our account

The first step is to register our account in the AWS Marketplace. Although the access will be made with our root user (or with a user to whom we have given permissions), we must complete the registration process as if it were another service.

The registration process will be carried out from the main cover of AWS Marketplace .

To register, we must press the button “Register as a seller in the Marketplace”, being logged in with our AWS account.

The first step will be to accept the terms and conditions of the Market, which we can download in PDF format for a more comfortable reading.

Although activating our account and accepting the terms is already possible to start selling, they will only allow us to give away products, since in order to be able to charge for them, we must complete the financial information.

In addition, the advisable thing to inspire confidence is to also fill our profile, so that the people who buy our products can see information about us.

On the cover we will be notified that we can start uploading products, but they will only allow us to offer free products, since we have not indicated the banking and tax information.

In order to sell products on the AWS Marketplace, we must be citizens or companies from Europe or the United States, and have a bank account at a US bank.

This part of the process is the longest part, since Amazon must verify each one of the data that we indicate and, especially if we are European, the process can get a lot longer.

In my case, the entire process from the time of discharge until I was able to upload my first payment AMI, was delayed more than 15 days.

Once we start the registration process, the first step is to fill out the tax information form. We must indicate at all times truthful data (since it will be checked)

The first questions are aimed at determining what type of form we should fill out (it is different if you are a citizen or company, European or US, etc.

According to the answers of the previous step, they will ask us for more detailed information, in order to make the necessary invoices.

Once our data is indicated, we must make a signed declaration for the IRS (the US tax system)

After the declaration process, we will be asked to indicate our fiscal data to complete this first step.

With the fiscal information already complete, the next step is to indicate the account where AWS will make us the income of the money generated with the sale of products in its market.

The bank account that we indicate must be from a US bank. If we have indicated that we are European citizens (or companies), Amazon gives us the option of working with Hiperwallet , which can offer us a virtual bank account in the USA, with which we can proceed.

In any case, the process involves asking Amazon to send us an email with information, where they will send us a link to directly register our account in Hyperwallet.

After indicating all the steps, and once the verification process is finished (which may take a long time), we will be able to sell products in the Amazon marketplace.

17 January, 2019 by macklus on IAM

Create IAM’s user for SNS notification sending

One of the recommended ways to perform sending notifications with SNS , is to create a user of IAM (the AWS Identity Management System).

Thus, we will have a user with their access codes, configured only to be able to send notifications, which offers us greater security.

Only requirement for the creation of this user is to have already created our SNS topic and to point its RNA. If you have not created it yet, you can follow the entry “ Configure SNS to receive notifications ”

Access to the IAM console

First step is always to access the AWS console .

Once inside, in the search bar we will look for “IAM” to be able to access the identity console.

Creating an IAM policy

Creating an IAM user to send with SNS, step 1

Once we are in the IAM welcome screen, we will start creating a new Policy, clicking on the “Policies” link in the menu on the left.

Since our user has to have an applied policy that gives him permission, we will start creating this policy, so that the creation process is easier to see.

Within the default policies of IAM we can find already defined policies for most of the services, although we will define our own to restrict access to the user as much as possible.

Once in the policy screen, click on the “Create a policy” button

Creating an IAM user to send with SNS, step 2

When accessing the policy creation screen we will have two options:

Visual editor: to navigate through existing policies and choose the ones we want.
JSON: to directly indicate the permissions that we want to apply.

In our case, we are going to go directly to the JSON tab, since we will introduce the permissions in that way. The code that we must indicate is the following:

{
     "Version": "2012-10-17",
     "Statement": [
         {
             "Effect": "Allow",
             "Action": "sns: Publish",
             "Resource": "arn: aws: sns: eu-west-1: 1836171893615: myTopic"
         }
     ]
 }

In the Resource section we must indicate the arn of the topic that we are going to use for the shipments.

By clicking on Review policy, it will be verified that the code we have set is correct, and we will go to the verification screen.

Creating an IAM user to send with SNS, step 3

Last step is to indicate a name for the policy, and a brief description, and clicking on “Create a policy”, we will have our policy created.

Create user on IAM

Creating an IAM user to send with SNS, step 4

On left menu we will see the different options, and we will access the users tab by clicking on the “Users” link.

Inside the user screen we will see the active users (if any), and we can create a new user by clicking on the “Add user (s)” button.

Inside the user creation screen, first we will indicate the name we want it to have, and that we will access it in a programmatic way.

Creating an IAM user to send with SNS, step 5

With basic user data already created, the next step is to add the permissions that you are going to have. As we have already created a policy, we just have to indicate that our user must use it, and for that:

Click on the button “Directly associate existing policies”
We use the search engine to locate the policy that we have just created.
Mark the policy or policies that we want add.

Creating an IAM user to send with SNS, step 6

Once finished, we click on the labels button to go to the labels screen.

Here we can indicate the labels that we want (it is optional), to identify the characteristics of that user.

Creating an IAM user to send with SNS, step 7

Once we finish, we will click on the “Next: review” button to finish.

Last step to create the user is to review the configuration of this and click on the “Create user” button.

If everything went well, we will see a message indicating that we have already created the user, and we can download or see the keys to connect with that user.

Creating an IAM user to send with SNS, step 9

After that, we can use this data to send SNS notifications.

10 January, 2019 by macklus on SNS: Simple Notification Service

Configure SNS to receive notifications

SNS (Simple Notification Service), is the product of Amazon Web Services for sending notifications. If you want to know more about this service you can read this post about AWS SNS.

To be able to use this service we have to understand what the topics are and the subscriptions. In SNS we create communication channels (topics), to which the person who wants to receive these notifications (subscribers) is connected. The concept is similar to the mailing lists (which would be our themes) and the people who sign up to receive the emails (which would be the subscribers).

The main advantage of SNS is that we can connect all types of services to a notification channel. For example, we can do that with every notification:

An email or SMS is sent.
A GET or POST request is made to a remote server.
An AWS Lambda is run.

Set up a notification in SNS

Create an SNS theme and make a subscription (in our example, an email account) is very simple by following these steps:

First we will access the AWS console , and from the drop-down menu Services we will choose (or look for) SNS.

If we see the welcome cover of SNS, we will click on “Get started” to access the work screen.

On the main screen, we select “Create topic” to create a new topic.

Now, we must indicate the name and a brief description (up to 10 characters) for our theme, and click on the “Create topic” button.

With our theme already created, we have to add subscribers that receive the messages that we send.

For this, once inside our subject, we will click on “Create subscription”, to add a new subscriber.

When clicking, a window will open where we must indicate 3 options:

The AWS ARN for our theme (which is already indicated)
The way we want to receive notifications (we will choose Email)
Destiny (in the mail case, the mail account we want to send notifications to

It is important to bear in mind that, if we have chosen the email, we must confirm the email address, by clicking on the link of the confirmation email sent to us. Until we do, we will not receive notifications and we will appear as “PendingConfirmation”.

Our email account is awaiting confirmation.

We will receive an email similar to this, with a link to confirm the subscription.

When clicking on the confirmation link, it will take us to a page similar to the one above, where it will confirm that the process has gone well.

By refreshing page, we’ll see our confirmed account, and we can receive notifications.

Send notifications to a topic

Once we have already set up our theme to be able to send notifications, we can see how it works by sending a trial.

To do this, in the configuration window of our theme, click on the “Publish to topic” button.

By clicking, you will be asked to indicate the subject of the notification, the content (we can indicate that it is raw and write any message), and clicking on “Publish message” will be sent to all subscribers.

In addition, we can use the AWS API for SNS to schedule the Shipments to SNS from our program, instances, lambdas, etc.

27 December, 2018 by macklus on EC2 cloud computing

FTP from PHP between instances of EC2

Recently I had to configure the FTP connection between 2 instances of EC2, which can be quite complicated if we do not consider several points.

The main thing is to understand that the connection between 2 instances of EC2 is a connection that is made between 2 servers protected by a firewall, which also probably work with an internal IP, and use a public IP associated with your VPC.

Problems on FTP’s protocol

Most FTP problems are design flaws of the protocol itself (due to their age), and can not be easily solved.

When we work with FTP in this type of environment we have to take into account two important aspects:

Passive mode

To connect between machines protected by firewalls, passive mode is used, where the ports of the connection are indicated directly.

In the instance that contains the FTP server, we must create a Security Group that allows connection to a high range of ports (for example, from 10,000 to 15,000).

Also, in the FTP server that we are using, we must configure the same range of ports as ports to be used for passive connections, so that all passive connections are made against the ports we have opened.

On the client side, we must indicate that the connection is passive, and depending on how we are connecting, we may have to send a PORT command indicating the public IP.

The connection IPs

When a passive connection is made, it is the client who initiates the data connection with the server.

When the client is going to connect, and depending on the way to do it, and the type of connection or VPC that we are using, it is possible that it sends the private IP instead of the public IP, so the communication with the server will not be possible.

This can be solved by forcing the public IP address with a PORT command, just after starting passive mode.

FTP connection

Although the connection from PHP is in principle exactly the same as any other type of connection, in PHP we can do it in a simpler way with the functions that the language itself incorporates.

En el siguiente ejemplo podéis ver un script completamente funcional de conexión en PHP:

<?php
ini_set('display_errors', 1);
ini_set('track_errors', 1);

$conn_id = ftp_connect('54.54.54.54');
ftp_login($conn_id, 'username', 'password');

ftp_set_option($conn_id, FTP_USEPASVADDRESS, false);
ftp_pasv($conn_id,true);

$l = ftp_nlist($conn_id, ".");
if( ! $l ) {
        print_r(error_get_last());
}
var_dump($l);

The two important instructions in the are those that use the functions ftp_set_option and ftp_pasv , which is where we indicate the way we are going to make the connection.

The option FTP_USEPASVADDRESS what it does is ignore the IP that the server tells us (which may be incorrect) and use the IP of the connection directly (which is the correct one).

The ftp_pasv option initiates a normal passive connection.

With these two options, it is possible to connect without problems even if we are connecting between instances separated by a firewall and using NAT to connect.

12 December, 2018 by macklus on Hosting

Should I change to a VPS?

Those of us dedicated to this web, with time accumulate small web that we maintain in different providers of cheap accommodation. They are products for which we do not charge much because they do not require much maintenance either, and they simply “are there”.

Over time, and as the number of websites we keep increasing, we get the million dollar question: Should I switch to a VPS?

The advantages of having a VPS

If you have an average number of websites (say 8 or 10), having them hosted on your own VPS gives you a series of undeniable advantages:

Total cost less than using separate accommodations.
Better use of resources, since the resources that a web does not use are available for others.
Greater ability to adapt the configurations to our liking.
All resources of the VPS are at our disposal, not as in normal shared accommodation.
The Internet is full of manuals to do the things we need.
Better options to expand resources if we are small.

The disadvantages of having a VPS

When you start using your own VPS, you realize that all that glitters is not gold, because having your own VPS also has disadvantages:

It is essential to have knowledge in systems administration.
If you do not want the pay high price, you will use a GNU / Linux VPS, which forces you to know about GNU / Linux
Normally you will use a control panel (such as Cpanel, Plesk or VestaCp), which may have an associated cost.
You must know and manage that control panel to avoid problems.
If you want to have total freedom, you can not use a control panel, but you must be a good administrator.
You have to take care of things like backup copies, save the logs, etc.

Then what do I do?

The main problem with using a VPS is that you have to know how to handle it. If you are willing to learn to do it, you will find many advantages. If you do not have time or desire, better not even start because in the end you will regret it.

If your motivation is just to save money, the safest thing is that you get fed up with the pileup. If you see it as a way to learn, improve and optimize, it’s not a bad idea.

Which VPS provider do you recommend?

I work mainly with 3:

Digital Ocean : For small VPS customers. You can earn $ 100 credit for 60 days using my referral link .
Time4VPS : I use them for VPS backups, monitoring, logs, etc. If you register, please use my referral link .
Amazon AWS: For more complete structures, static WordPress in high availability, etc.

All are cheap, and all are good, but do not stop comparing before deciding.

8 December, 2018 by macklus on Basic concepts

What is AWS SNS?

Inside the Amazon Web Services, SNS is one of two services that deal with sending notifications.

SNS stands for Simple Notification Service, and is the name that best defines its function: sending notifications to subscribers quickly and reliably.

How does SNS work?

SNS is a completely unattended and managed service, which operates in high availability, and in a safe and durable manner.

The main advantage of SNS is that we do not need to know anything about its structure or operation, since it is designed so that our only job is to integrate our application with the SNS API , and SNS is responsible for all message distribution work.

In its most basic structure, we only need to create a Notification Theme (a subject on which we want to send messages, such as “Errors”, to warn of system errors), and to subscribe the points that will receive the notification. These points can be of different types:

HTTP
HTTPS
Email
Email-JSON
Amazon SQS
Applicación
AWS Lambda
SMS

What are the costs of SNS?

SNS is a service designed for the mass sending of messages that, like the rest of AWS services, has a very low cost, based on the volume of messages sent.

In addition, the free AWS layer for SNS includes the first million messages, which will be more than enough for basic uses of the service.

It is important to clarify that the sending of messages via SMS if it has a different cost (is charged for each SMS sent), due to the characteristics of this type of service.

20 November, 2018 by macklus on Web pages

Your connection with this website is not completely secure?

If you have a web page (usually a WordPress) that works with SSL certificates (and if you do not have it immediately), you may find this error that, although it does not prevent our pages from being seen, may cause some elements they do not look good, or we get diverse errors.

Also, if you click on the exclamation that is in the address bar of the browser, you will see an image like the one shown below:

What is the error?

The error appears because, although our website is secure, some of the elements it contains are not.

Within a secure web page (which starts with https: //), all the elements have to be secure. If we have inside a web with https links to javascript, css or images that carry http: //, we will see the error.

How do I see where the problem is?

The easiest option is, within the page that gives us the problem, open a developer console, (usually just press F12 to appear).

Within the console tab of this, we will see messages indicating which resources are those that are incorrectly configured, in order to change them.

We show you an example in the following image:

Now, we just have to change those links in our template so that the error disappears.

Trick to be able to use http or https indistinctly

If we want to avoid this kind of problems, the best way is to use links that only contain a double bar.

For example, for an image:

The browser will use http or https as appropriate, or we will not have to worry about changing it by hand.

16 November, 2018 by macklus on Basic concepts

Create an Amazon AWS account

The process of creating an account in Amazon AWS is very simple, and includes a very innovative verification system that can give us some headaches.

For those who want to create an account, here we show you the steps, which begin by opening the address in our browser:

https://aws.amazon.com

From here, in the upper right corner, we will see an orange button that says “Create an AWS account”, that we must press.

The first thing Amazón AWS asks us is the data that we will use to access our account, which must be related to a valid email account.

In addition, we will have to indicate the name of the account that we want to use, which must be unique (usually it will be our name or that of the company).

After indicating the data of the account, they will ask us for our personal data, used for contact and billing.

It is important to note that the address field must be indicated in the form Street, PO Box, Company name, and in the lower field will be where we indicate the characteristics of the street (Floor 4, door 2, north staircase, etc). We do not have very clear the reasons for this, being that afterwards they ask us again for the postal code, but we must be alert in case it could give problems with the billing information.

Remind you that the data must be real and reliable, to avoid legal problems.

NOTE: We must have the telephone number that we indicate by hand, in order to make the verification.

The next step is to indicate a credit card (which has to be valid and be active), although that does not mean that we will incur any charge (they will not do so if we do not exceed the limits of the Amazon AWS free layer).

We must also indicate if we want to use our contact address for the realization of the invoice, or if we want to indicate new data.

Now that Amazon has our data, they will verify the phone number through a very interesting system.

First, they will show us a code on the screen, and then they will call us at the telephone number we have indicated, and we will have to enter the code with the telephone keypad.

The last step we must complete is to indicate the type of support we want. Although it can lead to confusion, no service plan is indicated here, only the support plan (the speed and way in which Amazon will solve the possible problems that we have).

In general, we will indicate the Free Plan, since it is sufficient for the normal use of our accounts.

With this, we have our Amazon AWS account created, and we will be redirected to the access page, where we can access with the created data.

Create AWS account, step 8 — Create AWS account, step 7

Once we have logged in, we can see in the alert area that we have access to the free AWS layer.

9 November, 2018 by macklus on Basic concepts

What is the cloud?

Although the concept of Internet cloud has been with us for almost 20 years, it has not been until relatively recently that it has begun to reach all areas, it is now common for cloud services to be offered in environments that are not technical in nature, where before it was impossible.

But … What the hell is “The Cloud”?

Understanding the concept Cloud …

What is meant by the Cloud is no more than many computers working together and providing services on the Internet. It’s that simple. It is not a new concept (although technically it is more advanced) nor is there anything special about it. The Cloud are only the computers of others.

Going into more detail, the cloud concept is a step forward in order to have resources that were previously unthinkable, in much more advantageous conditions, and above all in a much simpler way, which has facilitated its progress.

The Cloud is …

A more optimal cost: By paying only for what you use, the costs are reduced, since you can expand or reduce the services you use if you need them.
Dynamism: The resources that we use in the cloud can adapt according to our interests, even automatically. Thus, we can adapt to higher workloads without our service suffering, or reduce resources (and cost) in times of less work.
Ease: All Cloud services are designed to be used by people who do not have to have a very technical profile. Most services can be configured and adapted with a normal web browser.
Security: When using standardized services and managed by specialized providers, its use is safer. In addition, the Cloud even allows us to replicate our services among several countries, in the prevention of major catastrophes.

The fundamental basis of the Cloud is to optimize Internet resources, make them easier, accessible and safe even for the most complex processes, saving costs and energy in the process.

The Cloud is not …

A program that we sell to keep our files.
A web page to manage our invoices.
A dedicated server or VPS that charge us for hours.

While it is true that these services can be hosted in the cloud, when they offer us they are not offering us to work in the cloud, they only offer us a service that may or may not be related to it.

When the concept of Cloud began to become popular, there were many suppliers who wanted to get on the bandwagon to make money with a concept that was still not well understood. The term Cloud became one of the most important SEO terms, although in reality what they offered was not a real cloud, but the same infrastructures that were already available before. Luckily, little by little the concepts are clarified and it becomes more difficult to sell something that is not.

If you want to know more about the Cloud, you can go through the pages of Amazon AWS or Google Cloud, which are the main references of computing in the cloud.

Macklus.Net

Lasciate ogni speranza, voi ch'entrate