All codes above are fake but you should replace them with your own.
Nick
· 9 months ago
Actually this is an offtopic comment. But this is the only place I am able to contact you.
Your Arkayne "Contact us" form doesn't work, showing strange {message}} message. "Contact us" form on Aware Labs doesn't work too, showing "SMTPDataError at /contact/ (503, '5.0.0 Need RCPT (recipient)')".
I believe that you want to want to listen to your users/clients but something went wrong.
WBR, Nick
Rajesh D
· 9 months ago
If your server gets compromised, this script gives the hacker full access to your EC2 account -- private certificates, AWS secret key, and all. If that's a concern, one could scp a version of this script to the production EC2 instance and execute it only when new application and configuration updates are deployed to production after which the script could be deleted. That coupled with employing EBS for all persistent storage (and using EBS snapshots for daily data backups) is worth considering. EBS volume snapshots can be taken from outside of AWS and they don't require your AWS secret key -- so the script that takes those data snapshots doesn't even have to reside on the EC2 instance.
Nick
· 9 months ago
Please read blog post carefully all codes are fake.
Rajesh D
· 9 months ago
Sorry, I didn't explain my point clearly enough. I didn't mean to say that you are exposing your own codes (I understand fully that the codes you have shown above are made up.)
What I meant is that when you do deploy this script on to your server for real, you will have to put in real AWS codes into this script for it to work. At that point, a potential server compromise leaves a hacker with plain access to these codes and hence to your AWS account.
pkenjora
· 9 months ago
Rajesh,
Very valid point, I'm not sure its avoidable though. To back up to S3 the script needs the codes. Even if you do the backup manually your history log will show the keys in the command. I guess you could clear your history but does anyone do that?
Plus if your server is compromised, the hacker could simply wait until you back up again then cache the commands and codes. So Im not sure this problem is solvable, if the server is compromised then there is no way to avoid risk.
Rajesh D
· 9 months ago
If you use bash, it's very easy to disable writing of your command lines to the history file when you log out of ssh (basically, you would use "unset HISTFILE" in your bashrc.) There's also a command to remove your history from memory. I have this and some other tweaks on all my EC2 boxes.
You're right that a hacker could wait for you to use your secret keys/certs and grab them at that time. It's still a bit more reassuring not to have to keep your secret keys right on the server all the time.
Jonathan Fine
· 9 months ago
'sturday' should be 'saturday'.
Odzyskiwanie Danych
· 8 months ago
MS Windows taught me to back up. All those crasher back in the old day meant that if I wasn't saving every couple minutes I'd probably lose a lot of stuff. Now I'm backuping every piece of data I have.
danb
· 8 months ago
very helpful! thanks!
missing a line break on that first line.. import should be on its own line... and saturday is misspelled.
automated production
· 6 months ago
are these true? cause one here posted the codes here are fake, its good i haven't tried it.
Press Controls
· 6 months ago
Check your spelling.. but nice post.. this will me alot.. hope this codes are not fake...
Conrad Decker
· 5 months ago
Hi, thanks so much for this post - it's come in very handy. I'm curious though if anyone else sees sporadic results with this script? Sometimes it works, and sometimes it bombs on me. When it does bomb, I get the following error from my cronjob...
------ rm -f /mnt/saturday* /usr/local/bin/ec2-bundle-vol ..... Copying / into the image file /mnt/saturday... Excluding: /sys /proc /proc/sys/fs/binfmt_misc /dev /media /mnt /proc /sys /mnt/saturday /mnt/img-mnt 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00216 seconds, 485 MB/s mke2fs 1.39 (29-May-2006) ERROR: execution failed: "rsync -rlpgoD -t -r -S -l --exclude /sys --exclude /proc --exclude /proc/sys/fs/binfmt_misc --exclude /dev --exclude /media --exclude /mnt --exclude /proc --exclude /sys --exclude /mnt/saturday --exclude /mnt/img-mnt -X /* /mnt/img-mnt 2>&1 > /dev/null" /usr/local/bin/ec2-upload-bundle .... --manifest has invalid value '/mnt/saturday.manifest.xml': File does not exist or is not a file. Try 'ec2-upload-bundle --help'
----------
I've removed some of the command params, but you should get the idea. It seems to me that for some reason the manifest.xml file isn't being generated prior to trying to upload it to S3. Any thoughts?
Thanks in advance for any help!
Conrad Decker
· 5 months ago
Forgot to mention, that I only get that error when it runs from Cron. Anytime I run the script directly on the command line it seems to work fine. Very strange...
Dominic Watson
· 4 months ago
Interesting stuff. I did something similar with a bash script but not as a scheduled task (script would take image name as a parameter and then do the leg work).
I am personally keeping all data, logs and application config files on an EBS Volume and only run this bash script when I make changes to the OS. This way I plan to make backups with snapshots of the volume and/or copies of the data to S3.
Does this sound reasonable?
arpitjain
· 1 month ago
doesn't ec2 allows images of max size 10 Gb for bundling volumes ? That sure can't be enough for bundling everything. A better way would be to store your data on a different directory on ebs and bundle just the system files
Ezra Klughaupt
· 4 weeks ago
Thanks for the excellent script, but please do not ever run a cron at midnight, especially not in a shared environment like EC2. Most networks have surges at precisely that time because of all the cron scripts running. Better off picking an unusual time like 1:37. Midnight is far from the lowest traffic time anyway, most sites minimum is at 4-5 in the morning.
All Comments
Why are you not simply using a shell script if you just want to run a few shell commands? I think in this case it would make the code even simpler.
access_key = '8459945JFG8FDGJ38233'
secret_key = 'h9rtnretlfkgdnfgg843twlejrktjwlktwekl'
Your Arkayne "Contact us" form doesn't work, showing strange {message}} message.
"Contact us" form on Aware Labs doesn't work too, showing "SMTPDataError at /contact/
(503, '5.0.0 Need RCPT (recipient)')".
I believe that you want to want to listen to your users/clients but something went wrong.
WBR, Nick
What I meant is that when you do deploy this script on to your server for real, you will have to put in real AWS codes into this script for it to work. At that point, a potential server compromise leaves a hacker with plain access to these codes and hence to your AWS account.
Very valid point, I'm not sure its avoidable though. To back up to S3 the script needs the codes. Even if you do the backup manually your history log will show the keys in the command. I guess you could clear your history but does anyone do that?
Plus if your server is compromised, the hacker could simply wait until you back up again then cache the commands and codes. So Im not sure this problem is solvable, if the server is compromised then there is no way to avoid risk.
You're right that a hacker could wait for you to use your secret keys/certs and grab them at that time. It's still a bit more reassuring not to have to keep your secret keys right on the server all the time.
missing a line break on that first line.. import should be on its own line... and saturday is misspelled.
------
rm -f /mnt/saturday*
/usr/local/bin/ec2-bundle-vol .....
Copying / into the image file /mnt/saturday...
Excluding:
/sys
/proc
/proc/sys/fs/binfmt_misc
/dev
/media
/mnt
/proc
/sys
/mnt/saturday
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00216 seconds, 485 MB/s
mke2fs 1.39 (29-May-2006)
ERROR: execution failed: "rsync -rlpgoD -t -r -S -l --exclude /sys --exclude /proc
--exclude /proc/sys/fs/binfmt_misc --exclude /dev --exclude /media --exclude /mnt
--exclude /proc --exclude /sys --exclude /mnt/saturday --exclude /mnt/img-mnt -X
/* /mnt/img-mnt 2>&1 > /dev/null"
/usr/local/bin/ec2-upload-bundle ....
--manifest has invalid value '/mnt/saturday.manifest.xml': File does not exist or
is not a file.
Try 'ec2-upload-bundle --help'
----------
I've removed some of the command params, but you should get the idea. It seems to me that for some reason the manifest.xml file isn't being generated prior to trying to upload it to S3. Any thoughts?
Thanks in advance for any help!
I am personally keeping all data, logs and application config files on an EBS Volume and only run this bash script when I make changes to the OS. This way I plan to make backups with snapshots of the volume and/or copies of the data to S3.
Does this sound reasonable?