Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1) Added new code under linux-tools , this code can be usefull for b… #122

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions linux-tools/autotest-build-tools/FYI/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#==========================================================================================================
##
## FILE: README.txt
##
## DESCRIPTION: This File contains the steps of build and test autotest tests for all Linux Destro
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace Destro to Distribution , where ever mentioned

##
## OPTIONS: ---
## REQUIREMENTS: ---
## BUGS: ---
## NOTES: ---
## AUTHOR: IBM LTC Test Team
## COMPANY: IBM
## VERSION: 1.0
## CREATED: 16-Feb-2017 Abhishek Sharma < [email protected] >
## REVISION: 22-Feb-2017 Abhishek Sharma < [email protected] >
##==========================================================================================================


Supported Linux Destro : Redhat,Ubuntu,suse,CentOS
Supported Arch : Intel X86_64 and ia32
IBM Power (ppc,ppc64,ppc64le,ppcnf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure it builds for ppcnf?

IBM Z box( s390x)

Below steps are automted in run.py script, for better understanding steps are written down.

1) Git clone the latest autotest and autotest-linux-tools
2) Based on the arguments build the required packages and copy to autotest-linux-tools
3) Run the Regression test.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modify Step 2 autotest-linux-tools to autotest-client-tests/linux-tools
Step 3 has to be Run autotest-client-tests



NOTE : Before running the run.py script please verify modules/build_conf.py script
This is a configuration file to execute run.py
Before executing modify the data based on your requirement
Binary file added linux-tools/autotest-build-tools/FYI/chromedriver
Binary file not shown.
43 changes: 43 additions & 0 deletions linux-tools/autotest-build-tools/FYI/sources.list
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu CHANGE_ME main restricted
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME main #restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu CHANGE_ME-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME-updates main #restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu CHANGE_ME universe
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME universe
deb http://archive.ubuntu.com/ubuntu CHANGE_ME-updates universe
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://archive.ubuntu.com/ubuntu CHANGE_ME multiverse
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME multiverse
deb http://archive.ubuntu.com/ubuntu CHANGE_ME-updates multiverse
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu CHANGE_ME-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu CHANGE_ME-backports main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu CHANGE_ME-security main restricted
deb-src http://security.ubuntu.com/ubuntu CHANGE_ME-security main restricted
deb http://security.ubuntu.com/ubuntu CHANGE_ME-security universe
deb-src http://security.ubuntu.com/ubuntu CHANGE_ME-security universe
deb http://security.ubuntu.com/ubuntu CHANGE_ME-security multiverse
deb-src http://security.ubuntu.com/ubuntu CHANGE_ME-security multiverse

4 changes: 4 additions & 0 deletions linux-tools/autotest-build-tools/input_file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
curl
checkpolicy
cracklib
git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add all packages into input_file which has tests inside autotest and are identified for ubuntu

Binary file not shown.
Empty file.
50 changes: 50 additions & 0 deletions linux-tools/autotest-build-tools/modules/build_conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/usr/bin/python

#########################################################
#
# THIS IS BUILD CONF FILE , MODIFY BASED ON YOUR NEED
#
#########################################################

class generic_conf:
build_type = "UBUNTU" # Give CUSTOM/UBUNTU/CENTOS/RHEL for respective build
basedir = "/var/tmp/ubuntu_build" # Change this baseed on your requirement
logdir = "%s/logs" %basedir
logfile = "%s/final_report.logs" %logdir
package_list_file = "/home/ubuntu/autotest-build-tools/input_file" # This file will contain the package name to be build
binary_src_dir = "%s/autotest-binaries" %basedir # All the compile binaries will be copied in this location, you can change based on your requirement
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to include the build_src_distribution in the DIR structure as we can easily differentiate the builds based on versions.

sms_notofication = "n" # 'n' for no, this feature is only available for INDIA
mobile_number = "" # Only applicable for india number
python_modules = [ 'pyvirtualdisplay','selenium',]



class ubuntu_conf:
build_src_distribution = ['xenial'] # for rhel or suse mentioned the distribution name or distribution version rhel7.4 for building
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a single run tests can be built for only one release , so make this var as a single one instead of list

#build_src_distribution = ['zesty'] # for rhel or suse mentioned the distribution name or distribution version rhel7.4 for building
supported_release = [ 'precise','trusty','xenial','yakkety','zesty' ]
prerequisite_packages = ['packaging-dev','python-pip'] # If you want to add more packages to be part of your prerequisite add like ['pkg1','pkg2']
repo_file = "/etc/apt/sources.list" # Default repo file for ubuntu
template_repo_file = "FYI/sources.list" # Update your src repo entries in case default data is not enough to build the packages
chroot_path = "/var/cache/pbuilder"
pbuilder_script_file = "/usr/lib/pbuilder/pbuilder-buildpackage" # Do not change the file or else build is fail for all packages
pbuilder_build_path = "/var/cache/pbuilder/build/*/build/*"



class rhel_conf:
build_src_distribution = [] # for rhel or suse mentioned the distribution name or distribution version rhel7.4 for building
supported_release = [ '7.1','7.2','7.3' ]

class suse_conf:
build_src_distribution = [] # for rhel or suse mentioned the distribution name or distribution version rhel7.4 for building
supported_release = [ ]


class custom_conf:
#build_tag = [ 'ALL' ] # for specific build tag mentioned like mcp8_1-ppc64le
build_tag = [ '','' ] # for specific build tag mentioned like mcp8_1-ppc64le
input_file = "" # This file will contain the packges to be build for autotest
build_supporting_tags = ['','','','ALL']
sleep_tag = "3600" # After instance will sleep for these many sec, you can modify based on your requirement
num_of_build_per_cycle = "14"
Empty file.
65 changes: 65 additions & 0 deletions linux-tools/autotest-build-tools/modules/custom_module.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/python
import os,sys,logging,time


# Import custom modules
from build_conf import *
from generic_module import *

####################################################
# Capture all variables based on build conf file
####################################################
SLEEP_TAG = custom_conf.sleep_tag
BUILD_NUMS = custom_conf.num_of_build_per_cycle
BASEDIR = generic_conf.basedir
LOGDIR = generic_conf.logdir
return_code = os.system("ls -ld %s >/dev/null 2>&1" %LOGDIR)
if return_code != 0:
create_status = os.system("mkdir -p %s >/dev/null 2>&1 " %LOGDIR)
if create_status != 0:
print col.red + "\nERROR: Failed while creating directory %s\n" %LOGDIR + col.norm
sys.exit(0)


LOGFILE = generic_conf.logfile
logging.basicConfig(filename=LOGFILE,format='%(asctime)s - [ %(levelname)s ] - %(message)s',level=logging.DEBUG)

try:
LOCAL_BUILD_TAG = custom_conf.build_tag
except:
display_message_fn("Seems like build_tag is commented or deleted in conf file","ERROR")
footer_fn(BASEDIR,LOGDIR,LOGFILE)
sys.exit(0)

try:
BUILD_SUPPORTED_TAG = custom_conf.build_supporting_tags
except:
display_message_fn("Seems like build_supporting_tags is commented or deleted in conf file","ERROR")
footer_fn(BASEDIR,LOGDIR,LOGFILE)
sys.exit(0)






####################################################
# Function to verify given input is valid or not
####################################################
def varify_build_fn():
for input_tags in LOCAL_BUILD_TAG:
if input_tags in BUILD_SUPPORTED_TAG:
continue
else:
display_message_fn("Given input tag \"%s\" is not supported right now" %input_tags,"ERROR")
footer_fn(BASEDIR,LOGDIR,LOGFILE)
sys.exit(0)



##########################################################
# Funtion to setup required environment
##########################################################

varify_build_fn()
footer_fn(BASEDIR,LOGDIR,LOGFILE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where can this be used?

164 changes: 164 additions & 0 deletions linux-tools/autotest-build-tools/modules/generic_module.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
#!/usr/bin/python
import os,sys,logging
from build_conf import *
#############################################
# Create a class for define colors
#############################################
class col:
red = '\033[1;91m'
blue = '\033[1;94m'
bold = '\033[1m'
under_line = '\033[1;4m'
yellow = '\033[1;93m'
norm = '\033[0m'
green = '\033[1;92m'
cyan = '\033[1;36m'



###################################################
# Function to display the logs file information
###################################################
def footer_fn(arg1,arg2,arg3,arg4,basedir,logdir,logfile,binary_path,total_count,success_count,skipped_count,failed_count):
l_hostname = arg1
l_dist_name = arg2
l_dist_ver = arg3
l_build_req = arg4
local_basedir = basedir
local_logdir = logdir
local_logfile = logfile
l_total = total_count
l_success = success_count
l_skipped = skipped_count
l_failed = failed_count
l_binary_path = binary_path

print """



======================================================================================
A U T O T E S T B I N A R Y B U I L D T O O L

HOSTNAME : %s
CURRENT DISTRIBUTION NAME : %s
CURRENT DISTRIBUTION VERSION : %s
BUILD REQUESTED FOR : %s



BASEDIR : %s
LOGDIR : %s
LOGFILE : %s
TEST BINARIES : %s


TOTAL PACKAGE TO BE BUILD : %s
SUCCESS BUILD : %s
SKIPPED BUILD : %s
FAILED BUILD : %s


=======================================================================================


""" %(l_hostname,l_dist_name,l_dist_ver,l_build_req,local_basedir,local_logdir,local_logfile,l_binary_path,l_total,l_success,l_skipped,l_failed)


#####################################################
# Function to display header of script
#####################################################
def header_fn(arg1,arg2,arg3,arg4):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no calls for header_fn!

l_hostname = arg1
l_dist_name = arg2
l_dist_ver = arg3
l_build_req = arg4
print """


==============================================================================================
A U T O T E S T B I N A R Y B U I L D T O O L

H O S T N A M E : %s
C U R R E N T D I S T R I B U T I O N N A M E : %s
C U R R E N T D I S T R I B U T I O N V E R S I O N : %s
B U I L D R E Q U E S T : %s



===============================================================================================


"""%(l_hostname,l_dist_name,l_dist_ver,l_build_req)



def display_message_fn(arg1,arg2):
TEXT= arg1
MSG= arg2
cmd = "echo %s | awk -v msg=\"%s\" '{printf(\"%%5s %%-100s %%10s\",\"\",$0,\"[ \"msg\" ]\")}'" %(TEXT,MSG)
data_to_display = os.popen(cmd).read()
if MSG == "ERROR":
print col.red + data_to_display + col.norm
logging.error(TEXT)
elif MSG == "OK":
print col.green + data_to_display + col.norm
logging.info(TEXT)
elif MSG == "INFO":
print col.bold + data_to_display + col.norm
logging.info(TEXT)
elif MSG == "WARN":
print col.cyan + data_to_display + col.norm
logging.warning(TEXT)
elif MSG == "SKIPPED":
print col.cyan + data_to_display + col.norm
logging.info(TEXT)
else:
print data_to_display



def install_python_module_fn():
LISTS = generic_conf.python_modules
for list in LISTS:
status = os.system("python -c \"import %s\" >/dev/null 2>&1" %list)
if status != 0:
display_message_fn("Python module %s is not installed, installing it.. please wait." %list,"WARN")
command_check = os.system("sudo pip install %s >/dev/null 2>&1"%list)
if command_check == 0:
display_message_fn("Successfully installed module %s " %list,"OK")
else:
display_message_fn("Failed while installing module %s , please check and rerun the script" %list,"ERROR")
sys.exit(0)
else:
display_message_fn("Python module %s is already installed" %list,"OK")


def send_sms_fn(num,text):
value = os.system("dpkg --list|grep google-chrome >/dev/null 2>&1")
if value != 0:
display_message_fn("To run this sms feature you should install google chrome","INFO")
sys.exit(0)


from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 800))
display.start()
number_l = num
message_l = text
chrome_path = "FYI/chromedriver"
driver = webdriver.Chrome(chrome_path)
driver.get("http://site24.way2sms.com/content/index.html")
driver.find_element_by_xpath("""//*[@id="username"]""").send_keys("8147894264")
driver.find_element_by_xpath("""//*[@id="password"]""").send_keys("Letmein")
driver.find_element_by_id("loginBTN").click()
driver.find_element_by_css_selector(".button.br3").click()
driver.find_element_by_id("sendSMS").click()
frame = driver.find_element_by_xpath('//*[@id="frame"]')
driver.switch_to.frame(frame)
driver.find_element_by_id("mobile").send_keys(number_l)
driver.find_element_by_id("message").send_keys(message_l)
driver.find_element_by_id("Send").click()
Empty file.
19 changes: 19 additions & 0 deletions linux-tools/autotest-build-tools/modules/src_dest_file.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/python
############################################################################################
#
# Description : This file will contain <source> and <destination> of all test binaries
#
# <Source> : Once build is done, what need to copied.
# <Destination> : Source need to be copied in a specfic autotest folder to test.
#
#############################################################################################


#############################################################################################
# This class contain the packages name and what need to copied and where need to be copied
#############################################################################################
class ubuntu_src_dest:
libffi6 = {'TEST_SOURCE': ['testsuite'],'DESTINATION': ['']}
libmnl0 = {'TEST_SOURCE': ['examples/genl/.libs','examples/netfilter/.libs','examples/rtnl/.libs/'],'DESTINATION': ['tests']}
libnetfilter-conntrack3 = {'TEST_SOURCE': ['utils/.libs/'],'DESTINATION': ['utils']}
parted = {'TEST_SOURCE': ['tests/.libs','tests','build-aux'],'DESTINATION': ['']}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only four packages ? Add all binary paths for packages mentioned in input_file
And this is not being used!

Loading