pgbackup: Backup PostgresSQL

Mediante barman (pgbarman) podremos automatizar los backups y restauraciones de bases de datos PostgreSQL

En CentOS, simplemente deberemos instalar el paquete desde EPEL y configurar unos mínimos (/etc/barman/barman.conf)

[barman]
barman_home = /var/lib/barman
barman_user = barman
log_file = /var/log/barman/barman.log
compression = gzip
configuration_files_directory = /etc/barman.d

También deberemos configurar acceso mediante claves SSH (sin contraseña) en ambas direcciones desde el servidor de barman al usuario de la base de datos de las instancias que queramos hacer backup.

La configuración de los backups los haremos en ficheros independientes en el directorio /etc/barman.d para mayor comodidad, por ejemplo /etc/barman.d/pgm.conf

[pgm]
description = "postgres master"
ssh_command = ssh postgres@192.168.56.29
conninfo = host=192.168.56.29 user=postgres
retention_policy_mode = auto
retention_policy = RECOVERY WINDOW OF 30 days
wal_retention_policy = main

Mediante el comando show-server podremos buscar en que directorio deberemos dejar los archivados:

# barman show-server pgm | grep incoming_wals_directory
	incoming_wals_directory: /var/lib/barman/pgm/incoming

Mediante el modulo de puppet eyp-postgres podremos configurar un archive command para hacer rsync desde la base de datos al servidor.

	class { 'postgresql':
		wal_level => 'hot_standby',
		max_wal_senders => '3',
		checkpoint_segments => '8',
		wal_keep_segments => '8',
		archive_mode => true,
		archive_command_custom => 'rsync -a %p barman@192.168.56.31:/var/lib/barman/pgm/incoming/%f',
	}

Una vez este todo configurado podremos hacer copias de seguridad mediante backup, por ejemplo:

# barman backup pgm 
Starting backup for server pgm in /var/lib/barman/pgm/base/20160415T165403
Backup start at xlog location: 0/3000020 (000000010000000000000003, 00000020)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup end at xlog location: 0/30000E0 (000000010000000000000003, 000000E0)
Backup completed

Supongamos que tenemos ya el backup y hacemos lo siguiente en la base de datos:

postgres=# insert into test values('fuckthesystem');
INSERT 0 1
postgres=# select * from test;
       txt        
------------------
 sakura
 enlargeyourpenis
 fuckthesystem
(3 rows)


postgres=# delete from test;
DELETE 2
postgres=# select * from test;
 val 
-----
(0 rows)

postgres=# 

Podremos ver el listado de backups disponibles mediante list-backup:

# barman list-backup pgm
pgm 20160415T165403 - Fri Apr 15 14:54:04 2016 - Size: 19.3 MiB - WAL Size: 0 B

Para poder hacer la restauración necesitaremos algunos detalles del backup, los veremos mediante show-backup:

# barman show-backup pgm latest
Backup 20160415T165403:
  Server Name            : pgm
  Status                 : DONE
  PostgreSQL Version     : 90216
  PGDATA directory       : /var/lib/pgsql/9.2/data

  Base backup information:
    Disk usage           : 19.3 MiB
    Timeline             : 1
    Begin WAL            : 000000010000000000000003
    End WAL              : 000000010000000000000003
    WAL number           : 0
    Begin time           : 2016-04-15 14:54:01.835645+02:00
    End time             : 2016-04-15 14:54:04.514398+02:00
    Begin Offset         : 32
    End Offset           : 224
    Begin XLOG           : 0/3000020
    End XLOG             : 0/30000E0

  WAL information:
    No of files          : 0
    Disk usage           : 0 B
    Last available       : None

  Catalog information:
    Retention Policy     : VALID
    Previous Backup      : 20160415T142747
    Next Backup          : - (this is the latest base backup)

Necesitamos el nombre del backup y el “Begin time” exacto. Procedemos primero a parar la base de datos:

# /etc/init.d/postgresql-9.2 stop

A continuación indicamos a barman que recupere desde dicho backup indicando el begin time mediante –target-time y el nombre del backup. Podemos

# barman recover  --target-time "2016-04-15 14:54:01.835645+02:00" --remote-ssh-command="ssh postgres@192.168.56.29" pgm 20160415T142747 /var/lib/pgsql/9.2/data
Processing xlog segments for pgm
	000000010000000000000001
	000000010000000000000002
	000000010000000000000003
	000000010000000000000003.00000020.backup
	000000010000000000000004
Starting remote restore for server pgm using backup 20160415T165403 
Destination directory: /var/lib/pgsql/9.2/data
Copying the base backup.
Copying required wal segments.
The archive_command was set to 'false' to prevent data losses.

Your PostgreSQL server has been successfully prepared for recovery!

Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.

WARNING: Before starting up the recovered PostgreSQL server,
please review also the settings of the following configuration
options as they might interfere with your current recovery attempt:

    external_pid_file = '/var/lock/subsys/postgresql-9.2'			# write an extra PID file

Una vez recuperado, simplemente deberemos levantar la instancia:

/etc/init.d/postgresql-9.2 start

Si nos conectamos, veremos que tenemos los datos en el momento de hacer el backup:

[root@pgm ~]# psql -U postgres
psql (9.2.16)
Type "help" for help.

postgres=# select * from test;
       txt        
------------------
 sakura
 enlargeyourpenis
(2 rows)

postgres=# 

Tags:

Ransomware Evolution | @CloudExpo #InfoSec #DataCenter #Security

Initially, we came across ransomware which exploited the entire system and just restricted you from interacting with your own device, later on requiring you to pay dollars if you want to go back and use your computer.

And then it started becoming obsolete because an end-user. People were asking themselves: “That is my computer, would I pay $100 for it? If I don’t really have data, I’d better format my PC and start all over again.” So, that strategy – locking access to computers, started becoming obsolete. What did the bad guys do? They realized that the previous strategy was only good when the data that computer was holding was valuable. So they started asking ransom for the data, and that’s what they’re doing now.

read more

Cloud-Phobia Bubble | @CloudExpo @AvereSystems #DigitalTransformation

Sometime over the last 10 years, the term “cloud” transformed from a big, fluffy collection of water droplets in the sky to the (somewhat daunting) future of IT and a fundamental part of some of the most innovative technologies in the world today. Cloud adoption rates have been increasing steadily over the years due to its compelling economic and technical advantages, allowing organizations to fulfill user requests instantaneously and only for the period of time that they need it. But for every organization that is fearlessly embracing cloud computing and the many benefits it offers, there is another organization that is hesitantly weighing infrastructure options, fearful of taking the leap.

read more

Cloud Expo Returns to the Javits Center | @CloudExpo #IoT #Cloud

This is not a small hotel event. It is also not a big vendor party where politicians and entertainers are more important than real content.
This is Cloud Expo, the world’s longest-running conference and exhibition focused on Cloud Computing and all that it entails.
If you want serious presentations and valuable insight about Cloud Computing for three straight days, then register now for Cloud Expo.

read more

Critical (Outdoor) IoT Applications | @ThingsExpo #IoT #IIoT #M2M #InternetOfThings

It’s safe to assume that the majority of all Internet of Things (IoT) devices operate near large populations of people. Of course, right? This is where the action happens – smart devices, smart cars, smart infrastructure, smart cities, etc. Plus, the cost of getting “internet-connected” in these areas is relatively low – public access to Wi-Fi is becoming widely available, cellular coverage is blanketed over cities, etc.

read more

[session] The Best Course Around Multi-Cloud Adoption By @Cloudyn_buzz | @CloudExpo #Cloud

Up until last year, enterprises that were looking into cloud services usually undertook a long-term pilot with one of the large cloud providers, running test and dev workloads in the cloud. With cloud’s transition to mainstream adoption in 2015, and with enterprises migrating more and more workloads into the cloud and in between public and private environments, the single-provider approach must be revisited.
In his session at 18th Cloud Expo, Yoav Mor, multi-cloud solution evangelist at Cloudyn, will examine the risks of sourcing IT resources from a single cloud service vendor, as well as the benefits to be reaped by adopting multi-cloud strategies.

read more

‘How to Create Angular 2 Clients for the Cloud’ Workshop at @CloudExpo | #Cloud

Angular 2 is a complete re-write of the popular framework AngularJS. Programming in Angular 2 is greatly simplified – now it’s a component-based well-performing framework. This immersive one-day workshop at 18th Cloud Expo, led by Yakov Fain, a Java Champion and a co-founder of the IT consultancy Farata Systems and the product company SuranceBay, will provide you with everything you wanted to know about Angular 2.

read more

Step-by-Step IoTization | @ThingsExpo #IoT #IIoT #M2M #DigitalTransormation

The paradigm has shifted. A Gartner survey shows that 43% of organizations are using or plan to implement the Internet of Things in 2016. However, not just a handful of companies are still using the old-style ad-hoc trial-and-error ways, unaware of the critical barriers, paint points, traps, and hidden roadblocks. How can you become a winner?
In his session at @ThingsExpo, Tony Shan will present a methodical approach to guide the holistic adoption and enablement of IoT implementations. This overarching model is an adaptive framework to systematically handle the rapid changes and exponential growth, composed of the following modules: Prototype, Extend, Assess, Strategize, Formulate, Accelerate, Initiate, and Revamp (PEAS FAIR). We will zoom in to individual components in each module. Working examples will be discussed to illustrate the practical use of this method in the real world, along with best practices and lessons learned to help organizations IoTize step-by-step.

read more

IBM announces four new clients for video business unit

Curved video wallIBM has revealed four new client wins for its video business, IBM Cloud Video, a couple of hours ahead of its quarterly earnings announcement.

Speaking at NAB Show, the company announced Comic-Con HQ, Canadian Broadcasting Corporation, AOL and Broadway Video will now all be utilizing the IBM video platform. The company expects the market to exceed $100 billion in the next three years, as well as digital video to account for 80% of all internet traffic by 2019.

“IBM is at the forefront of the industry at a time when video is the driving influence in how organizations communicate, share information, and entertain,” said Braxton Jarratt, General Manager of the IBM Cloud Video business unit. “Today’s announcements will be viewed as a significant milestone in the company’s cloud video strategy, as IBM makes the sharing, distribution, and management of video increasingly simple across any device.”

IBM announced the acquisition of Ustream in January though financials of the agreement were not disclosed. Ustream created a cloud model to support live and on-demand video streams and claimed to have 80 million viewers per month from customers including NASA, Samsung, Facebook, Nike and The Discovery Channel. The IBM Cloud Video business unit was formed by the combination of IBM’s R&D dollars alongside acquisitions of Clearleap, Ustream, Aspera and Cleversafe.

The deal with Comic-Con HQ will offer numerous services including subscriber and content management, billing, and video compatibility on multiple devices. The Canadian Broadcasting Corporation will be using IBM’s tech to support its next-generation, ad-supported streaming video service. AOL will be using transfer and automation software from Aspera (an IBM company) to power its media management platform.

The news comes ahead of the company’s quarterly earnings, in which analysts expect IBM to announce further revenue declines. The company has reported revenue declines for 15 straight quarters, though these trends have been witnessed by several tech giants who have been primarily associated with now-legacy IT, not only IBM. The move into cloud computing is seemingly one of a number of strategies set in place for IBM to counter negative growth, and carve a new niche in the digital ecosystem.

Microsoft catching up with Salesforce in SaaS revenues, argues research

(c)iStock.com/Nicolas McComber

Microsoft has Salesforce in its sights as the leading cloudy software as a service (SaaS) vendor, according to the latest note from analyst house Synergy Research.

According to the research, the overall enterprise SaaS market grew by almost 40% in 2015, with forecasts from Synergy suggesting it will more than triple in size within five years. The consumer SaaS market, on the other hand, is only a third the size of the enterprise with a lower growth rate.

Microsoft’s comparative annual growth of 70%, compared to Salesforce’s 21%, means that they have captured more than 10% of the overall market share. Adobe and SAP are the third and fourth largest vendors in the space respectively, with ADP, Google, IBM, Intuit, Oracle, and Workday all in the top 10.

“In many ways SaaS is a more mature market than other cloud markets like IaaS or PaaS. However, even for SaaS it is still early days in terms of market adoption,” said John Dinsdale, Synergy chief analyst and research director.

“It is notable that the big three traditional software vendors – Microsoft, Oracle and IBM – are all now growing their SaaS revenues faster than the overall market and yet SaaS accounts for less than 8% of their total software revenues.”

The moves into cloud software as a service from the longer standing players has been extremely well documented, from IBM ploughing billions of dollars into their cloud play, to similar grandstanding from SAP. Oracle CTO Larry Ellison argued last March that his company would sell more SaaS and PaaS new business than Salesforce in 2015, with Synergy describing Oracle as “surging ahead”.

A recent report from identity management provider Okta found that Office 365 continues to be the most popular cloud app used in the enterprise, ahead of Salesforce and Box, while organisations are increasingly using Office and Google Apps in tandem.

The cloud news categorized.