Attila Kanto akanto
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| get_mnist_example() { | |
| su -c "cd /home/spark && git clone https://github.com/akanto/ml.git && cd ml && git checkout hcc-cuda-blog" spark | |
| } | |
| main() { | |
| get_mnist_example | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| install_drivers() { | |
| cat > /etc/yum.repos.d/amzn-graphics.repo <<'EOF' | |
| [amzn-graphics] | |
| name=amzn-graphics-Base | |
| mirrorlist=http://repo.$awsregion.$awsdomain/$releasever/graphics/mirror.list | |
| mirrorlist_expire=300 | |
| metadata_expire=300 | |
| priority=9 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| auto_accept: True | |
| external_auth: | |
| pam: | |
| saltuser: | |
| - .* | |
| - '@wheel' # to allow access to all wheel modules | |
| - '@runner' # to allow access to all runner modules | |
| - '@jobs' # to allow access to the jobs runner and/or wheel module |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [hdfs@docker-ambari ~]# for i in 1 4 8 12 16 20 24 28 32 36; do | |
| > hadoop jar $JAR_JOBCLIENT TestDFSIO -Ddfs.replication=3 -write -nrFiles $i -fileSize 8192 | |
| > done | |
| WARNING: Use "yarn jar" to launch YARN applications. | |
| 16/04/11 14:03:40 INFO fs.TestDFSIO: TestDFSIO.1.8 | |
| 16/04/11 14:03:40 INFO fs.TestDFSIO: nrFiles = 1 | |
| 16/04/11 14:03:40 INFO fs.TestDFSIO: nrBytes (MB) = 8192.0 | |
| 16/04/11 14:03:40 INFO fs.TestDFSIO: bufferSize = 1000000 | |
| 16/04/11 14:03:40 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO | |
| 16/04/11 14:03:41 INFO fs.TestDFSIO: creating control file: 8589934592 bytes, 1 files |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| WARNING: Use "yarn jar" to launch YARN applications. | |
| 16/04/11 13:41:18 INFO fs.TestDFSIO: TestDFSIO.1.8 | |
| 16/04/11 13:41:18 INFO fs.TestDFSIO: nrFiles = 1 | |
| 16/04/11 13:41:18 INFO fs.TestDFSIO: nrBytes (MB) = 8192.0 | |
| 16/04/11 13:41:18 INFO fs.TestDFSIO: bufferSize = 1000000 | |
| 16/04/11 13:41:18 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO | |
| 16/04/11 13:41:19 INFO fs.TestDFSIO: creating control file: 8589934592 bytes, 1 files | |
| 16/04/11 13:41:19 INFO fs.TestDFSIO: created control files for: 1 files | |
| 16/04/11 13:41:19 INFO impl.TimelineClientImpl: Timeline service address: http://ip-10-0-71-101.node.dc1.consul:8188/ws/v1/timeline/ | |
| 16/04/11 13:41:19 INFO client.RMProxy: Connecting to ResourceManager at ip-10-0-71-101.node.dc1.consul/10.0.71.101:8050 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [hdfs@docker-ambari ~]# for i in 1 4 8 12 16 20 24 28 32 36; do hadoop jar $JAR_JOBCLIENT TestDFSIO -Ddfs.replication=1 -write -nrFiles $i -fileSize 8192; done | |
| WARNING: Use "yarn jar" to launch YARN applications. | |
| 16/04/11 12:31:39 INFO fs.TestDFSIO: TestDFSIO.1.8 | |
| 16/04/11 12:31:39 INFO fs.TestDFSIO: nrFiles = 1 | |
| 16/04/11 12:31:39 INFO fs.TestDFSIO: nrBytes (MB) = 8192.0 | |
| 16/04/11 12:31:39 INFO fs.TestDFSIO: bufferSize = 1000000 | |
| 16/04/11 12:31:39 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO | |
| 16/04/11 12:31:40 INFO fs.TestDFSIO: creating control file: 8589934592 bytes, 1 files | |
| 16/04/11 12:31:40 INFO fs.TestDFSIO: created control files for: 1 files | |
| 16/04/11 12:31:41 INFO impl.TimelineClientImpl: Timeline service address: http://ip-10-0-71-101.node.dc1.consul:8188/ws/v1/timeline/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "host_groups": [ | |
| { | |
| "name": "master", | |
| "components": [ | |
| { | |
| "name": "NAMENODE" | |
| }, | |
| { | |
| "name": "SECONDARY_NAMENODE" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| : << USAGE | |
| ################################## | |
| curl -L https://gist.githubusercontent.com/akanto/3bc6140a4aadece4677c/raw/7dc1661b3ce1e665e2bde82ba43f81c43d4e9bc4/docker-install | bash | |
| ################################## | |
| USAGE | |
| debug() { | |
| [[ "$DEBUG" ]] && echo "-----> $*" 1>&2 | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Hibernate: | |
| /* named HQL query Stack.findById */ select | |
| stack0_.id as id1_18_0_, | |
| resources1_.id as id1_14_1_, | |
| instancegr2_.id as id1_9_2_, | |
| instanceme3_.id as id1_10_3_, | |
| stack0_.account as account2_18_0_, | |
| stack0_.consulServers as consulSe3_18_0_, | |
| stack0_.credential_id as credent14_18_0_, | |
| stack0_.description as descript4_18_0_, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@tera-test-cent7-master3-5-1435741883353 ~]# docker logs 9618e34d64a6 | |
| ==> WARNING: It is highly recommended to set GOMAXPROCS higher than 1 | |
| ==> Starting Consul agent... | |
| ==> Starting Consul agent RPC... | |
| ==> Consul agent running! | |
| Node name: 'tera-test-cent7-master3-5-1435741883353' | |
| Datacenter: 'dc1' | |
| Server: false (bootstrap: false) | |
| Client Addr: 0.0.0.0 (HTTP: 8500, HTTPS: -1, DNS: 53, RPC: 8400) | |
| Cluster Addr: 10.0.189.12 (LAN: 8301, WAN: 8302) |
NewerOlder