Displaying System info in Python

[zak100]

Hi,

I am trying to display system information in Python.

I am now getting following error:

Quote

 

Traceback (most recent call last):

File "/home/zulfi/PycharmProjects/Classes/psutilPrintInfo.py", line 31, in <module>

obj_Counting_SysInfo.main()

File "/home/zulfi/PycharmProjects/Classes/psutilPrintInfo.py", line 27, in main

self.system_info()

File "/home/zulfi/PycharmProjects/Classes/psutilPrintInfo.py", line 16, in system_info

mem = process.get_memory_info()[0] / float(2 ** 30) # memory in GB

AttributeError: 'Process' object has no attribute 'get_memory_info'

 

 

My program is below:

import psutil
import os

class Counting_SysInfo:
    def __init__(self, i):
        self.i = i

    def Print_Numbers(self):
        for i in range(50):
            print (i)

    def system_info(self):
        # return the memory usage in MB
        self.Print_Numbers()
        process = psutil.Process(os.getpid())
        mem = process.get_memory_info()[0] / float(2 ** 30)  # memory in GB
        cpu = process.cpu_percent() / psutil.cpu_count()
        disk = process.disk_usage(part.mountpoint).percent  # https://github.com/giampaolo/psutil/blob/master/scripts/disk_usage.py
        vMem = int(process.virtual_memory().available)
        print(process.memory_info().rss)  # in bytes
        RSS = process.memory_info().rss
        # https://stackoverflow.com/questions/17990227/whats-the-unit-of-rss-in-psutil-process-get-memory-info
        pageFaults = process.memory_info().pfaults  # https://readthedocs.org/projects/giamptest/downloads/pdf/latest/
        print("mem= ", mem, "cpu= ", cpu, 'disk= ',disk, "VMem= ", vMem, "RSS= ", RSS, "page faults=", pageFaults )

    def main(self):
        self.system_info()

if __name__ == "__main__":
    obj_Counting_SysInfo = Counting_SysInfo(10)
    obj_Counting_SysInfo.main()

Somebody please guide me.

 

Zulfi.

[Strange]

You have:

42 minutes ago, zak100 said:

mem = process.get_memory_info()...

And:

42 minutes ago, zak100 said:

pageFaults = process.memory_info()...

One of them is wrong ...

[Ghideon]

54 minutes ago, zak100 said:

Somebody please guide me.

 

1: Error message says (bold by me):

Quote

AttributeError: 'Process' object has no attribute 'get_memory_info'

So get_memory_info does not exist in a process object. What happens if you replace get_memory_info with an attribute that exists?

2: Reading here: https://psutil.readthedocs.io/en/latest/#psutil.Process.memory_info gives some hints, "memory_info" might be useful. (Or use @Strange's suggestion above)

3: To reduce the risk that unnecessary details we test the change in isolation with three lines of code only. First repeat the error

import psutil
import os
print(psutil.Process(os.getpid()).get_memory_info()[0])

Output:

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    print(psutil.Process(os.getpid()).get_memory_info()[0])
AttributeError: 'Process' object has no attribute 'get_memory_info'

4: Try a modified version

import psutil
import os
print(psutil.Process(os.getpid()).memory_info()[0])

144... (example )

 

5: Try the whole program with the change above

 

 

You might want to study more about how to debug code. It may help in the long run to be able to solve the most basic issues without guiding.

 

 

 

 

 

Edited March 21, 2020 by Ghideon
x-post with Strange

[Strange]

12 minutes ago, Ghideon said:

1: Error message says (bold by me):

 

12 minutes ago, Ghideon said:

You might want to study more about how to debug code. It may help in the long run to be able to solve the most basic issues without guiding.

 

Good points. It is always a good idea to think about what the error message says and how it relates to the code you have written. Sometimes error messages are really unhelpful, but usually they give you a good clue to where the problem is.

Python error messages can be confusing because they show a traceback through the calls that called the program. Usually, the last line of code referenced ("line 16, in system_info" in this example) is where the problem is (occasionally, it might be that the problem is further back because of how the function was called or the parameters it was passed). And the very last line indicates the nature of the problem.

 

[zak100]

Hi,

Memory part worked, now I am getting error with disk usage:

 

import psutil
import os
print(psutil.Process(os.getpid()).disk_usage('/').percent)

Error is:

Quote

print(psutil.Process(os.getpid()).disk_usage('/').percent)
AttributeError: 'Process' object has no attribute 'disk_usage'

I found the following link:

https://pypi.org/project/psutil/

Somebody please guide me.

 

Zulfi.

[Strange]

Disk usage appears to be an attribute of psutil, not pustil.Process: https://psutil.readthedocs.io/en/latest/#psutil.disk_usage

[Ghideon]

15 hours ago, Strange said:

Python error messages can be confusing because they show a traceback through the calls that called the program. Usually, the last line of code referenced ("line 16, in system_info" in this example) is where the problem is (occasionally, it might be that the problem is further back because of how the function was called or the parameters it was passed). And the very last line indicates the nature of the problem.

Good that you pointed that out! (Reading the trace is one of those things I "just do" without thinking). Interpreting the error messages for python is similar to other environments, it is a useful skill to have in your "set of tools" when tracking down errors and bugs. Here is a link that may help https://realpython.com/python-traceback/.

 

31 minutes ago, zak100 said:

Memory part worked, now I am getting error with disk usage:

Good to know. 

31 minutes ago, zak100 said:

Error is:

Quote

print(psutil.Process(os.getpid()).disk_usage('/').percent)
AttributeError: 'Process' object has no attribute 'disk_usage'

 

If you apply what we said regarding the first problem above, do you see what is causing the error?
If you check the psutil documentation (for instance at the link you provided) what does it say about disk usage?
If you check the reference in your code (https://github.com/giampaolo/psutil/blob/master/scripts/disk_usage.py) how and why is that code different that yours?

edit: Please review your requirement; if your program had worked you would probably have got the disk usage for a single process, not the disk usage for a mount point (they may differ) 
 

Edited March 22, 2020 by Ghideon
x-post with Strange (I need a shortcut for that, happens a lot :-) )

[zak100]

Hi,

print(psutil.disk_usage('/'))

I got following answer:

Quote

sdiskusage(total=982900588544, used=141382160384, free=791518343168, percent=15.2)
12050432

Is the above okay, can I find it for a process? because I have to compare two processes.

print(psutil.Process(os.getpid()).cpu_percent(interval=1)

But its giving:
0.0 why? Please guide me.

For the following two I am getting attribute error:

print(psutil.Process(os.getpid()).virtual_memory().available)//no attribute
print(psutil.Process(os.getpid()).memory_info().pfaults)//no aatribute

 

Somebody please guide me.

 

Zulfi.

[Ghideon]

22 minutes ago, zak100 said:

Is the above okay, can I find it for a process? because I have to compare two processes.

Do you expect the free disk space to be depending on the process?

1 hour ago, zak100 said:

But its giving:
0.0 why? Please guide me.

If you perform a call to measure the CPU load from the process and the process performs no work then 0.0 seems normal?

[Ghideon]

1 hour ago, zak100 said:

print(psutil.Process(os.getpid()).virtual_memory().available)//no attribute
print(psutil.Process(os.getpid()).memory_info().pfaults)//no aatribute

Same as above. Read about memory_info available in process class. What is your requirement? The process class will get you the virtual memory used by that process. You seem to want to know how much free memory there is left?

 

[zak100]

Hi,

Thanks for your cooperation, I am still getting zero for cpu usage:

 

import psutil
import os
#import memory_usage

print("memory usage=", psutil.Process(os.getpid()).memory_info()[0])

print(psutil.disk_usage('/'))
print (psutil.Process(os.getpid()).cpu_percent(interval=1))
print("virtual mem= ",psutil.virtual_memory().available)
#print(psutil.Process(os.getpid()).memory_info().pfaults)
print("rss=", psutil.Process(os.getpid()).memory_info().rss)
print (psutil.Process(os.getpid()).cpu_percent(interval=1))

Please guide me how to get some non-zero for cpu utilization.

 

For page faults, following link says:

https://readthedocs.org/projects/giamptest/downloads/pdf/latest/

linix, its shared. I am working on linix.

 

Somebody please guide me about page faults and memory usage.

 

Zulfi.

 

 

[Ghideon]

30 minutes ago, zak100 said:

I am still getting zero for cpu usage:

If CPU usage is zero then a report of zero cpu is expected? If your process is single threaded and you make a blocking call, how much is the process using the CPU? What is the problem with zero of the CPU usage is zero?*

54 minutes ago, zak100 said:

Please guide me how to get some non-zero for cpu utilization.

What it is that you want your program to achieve? Do you want to generate CPU load, just to get a non-zero number?*

1 hour ago, zak100 said:

Somebody please guide me about page faults and memory usage.

Can you be more specific? Does the code give you unexpected output?

 

 

*) I'll post questions instead of code suggestions at this time. If I just post a program that runs there is a risk that it solves the wrong problem.

[Strange]

1 hour ago, zak100 said:

Please guide me how to get some non-zero for cpu utilization.

You would need to get the process to do some significant work to get a non-zero value for this. You had a previous thread about sorting, didn't you. So maybe get the program to sort a large list of numbers before getting the cpu utilisation.

 

1 hour ago, zak100 said:

For page faults, following link says:

https://readthedocs.org/projects/giamptest/downloads/pdf/latest/

linix, its shared. I am working on linix.

 

Somebody please guide me about page faults and memory usage.

The 'shared' value is the amount of memory that could be shared with other processes. Not the number of page faults.

Quote

shared: (Linux) memory that could be potentially shared with other processes. This matches “top“‘s SHR column).

https://psutil.readthedocs.io/en/latest/#psutil.Process.memory_info

It doesn't look as if you can get page faults in Linux. The 'dirty' value might be the closest thing you can get.

Note that several of the Linux fields refer to 'top'. You can find a description of those fields in the 'top' man page.

[zak100]

Hi and Good Morning,

 

CPU utilization information is still zero. I tried with the sorting code:

 

 def systeminfo(self):
        print("memory usage=", psutil.Process(os.getpid()).memory_info()[0])
        print(psutil.disk_usage('/'))
        print(psutil.Process(os.getpid()).cpu_percent(interval=1))
        print("virtual mem= ", psutil.virtual_memory().available)
        # print(psutil.Process(os.getpid()).memory_info().pfaults)
        print("rss=", psutil.Process(os.getpid()).memory_info().rss)
        self.selectionSort()
        print(psutil.Process(os.getpid()).cpu_percent(interval=1))


    def main(self):
        self.generate1000_5digit_RandomNum()
        self.systeminfo()
        
    if __name__ == "__main__":
    objSSSort = SSSort(1000)
    objSSSort.main()
   

I tried sorting with 1000 elements.

Quote

memory usage= 15269888
sdiskusage(total=982900588544, used=141417086976, free=791483416576, percent=15.2)
0.0
virtual mem=  12050067456
rss= 15269888
(0, 10000)

and at other place:

Quote

(998, 99919)
(999, 99927)
0.0

 

Somebody please guide me.

Zulfi.

Edited March 23, 2020 by zak100

[Strange]

6 minutes ago, zak100 said:

Hi and Good Morning,

 

CPU utilization information is still zero. I tried with the sorting code:

You need to read the documentation for this function:

Quote

Warning: the first time this method is called with interval = 0.0 or None it will return a meaningless 0.0 value which you are supposed to ignore. 

https://psutil.readthedocs.io/en/latest/#psutil.Process.cpu_percent

[Ghideon]

26 minutes ago, Strange said:

You need to read the documentation

Agree.

Here is an attempt at describing. Extract from code posted above.

self.selectionSort()
print(psutil.Process(os.getpid()).cpu_percent(interval=1))

First the sorting is performed. That means there is load on the CPU for a while.
Then there is a call to cpu_percent with the interval set to 1. If I got it right from the documentation that means the process waits (blocks) for one second and then reports the CPU usage during that second. Since sorting is already done and the process is not performing work other than the blocking call to cpu_percent then the expected CPU load is zero, correct? I would need to study this more, but your code seem to be intended for a process with multiple threads where other threads are doing work while one thread calls cpu_percent for measurements. 

(A longer explanation than the short comment I made above)

 

 

Edited March 23, 2020 by Ghideon

[Strange]

56 minutes ago, Ghideon said:

Agree.

But I failed to read the code: I hadn't noticed that the call has "interval=1". So my highlighting that warning was incorrect.

[zak100]

Hi,

Thanks for your interest in my problem. I have not explicitly created any thread. I have used the cpu-percent before and after psutil but I am getting zero in both the cases.

 

Somebody please guide me.

 

Zulfi.

Hi,

I am trying the selection with 1 million integers but my system hangs up.

I am using following following command:

if __name__ == "__main__":
    objSSSort = SSSort(1000000)
    objSSSort.main()

In the above code, I am trying to run my selection sort program with 1000000 integers
How to solve this problem in Python?
Is there any size limitation with integers in python?
Zulfi.

 

[Ghideon]

2 hours ago, zak100 said:

Thanks for your interest in my problem. I have not explicitly created any thread. I have used the cpu-percent before and after psutil but I am getting zero in both the cases.

Somebody please guide me.

You are still not clear about what you want to do or what you expect to happen. Have you yet read about blocking calls and how they work in this case?

Here is an ugly attempt*. I guess that you need cpu_percent(interval=1) to return non-zero for the current process. So I create one thread doing some work while sampling CPU load.

import psutil
import os
import random
import threading

pid=os.getpid() # current process

# define some meaningless task for the CPU
def sorting(): 
  print("sorting()")
  for v in range(100):
    sequence = [i for i in range(20000)]
    random.shuffle(sequence)
    sequence.sort();
  print("sorting() done")


print("start thread")
psutil.Process(pid).cpu_percent(interval=0)
th=threading.Thread(target=sorting,args=())
th.start()
while th.is_alive():
  print("CPU usage:"+ str(psutil.Process(pid).cpu_percent(interval=1)))
print("all done")

 

 

2 hours ago, zak100 said:

my system hangs up.

 The system did not bother to produce an error message? Out of memory error?

 

2 hours ago, zak100 said:

Is there any size limitation with integers in python?

>>>sys.maxint
9223372036854775807
(Which is 2^63 - 1) Not necessarily the same on all versions and on all systems.

Here is a link: https://lmgtfy.com/?q=Is+there+any+size+limitation+with+integers+in+python%3F

 

 

*) have limited time, this is a quick hack that runs, it is not necessarily giving any meaningful measurements.

 

Edited March 23, 2020 by Ghideon

[zak100]

Hi,

Thanks for your program. It looks that the key is creating another thread. I also tried with large values but did not create a separate thread. (interval=0) not important.

Quote

Have you yet read about blocking calls and how they work in this case?

They wait for input. Do you mean that the blocking call stops the execution of all processes and then returns the cpu time for a particular process?

 

 

Zulfi.

Edited March 24, 2020 by zak100

[zak100]

Hi,

Thanks, your trick succeeded. I have to make changes in the code. I have used inner function. Sorting code got disturbed. Its showing multiple output for cpu utlization. Somebody please correct my code.

import random
import psutil
import os
import threading

class SSSort:
    arr = []  # class var access by className.arr

    def __init__(self, n):
        self.n = n

    def generate1000_5digit_RandomNum(self):
        for i in range(self.n):
            num = random.randint(10000, 99999)
            SSSort.arr.append(num)

        for j in range(self.n):
            print("random {0}".format(SSSort.arr[j]))

    def cpu_util_info(self):
        k= self.n
        arr2 = SSSort.arr[:]
        def find_the_min_element(k, i, arr2):
           min = i
           for j in range(i+1, k):
              if (arr2[j] < arr2[min]):
                 min = j
           return min

        def selectionSort(k, arr2):
            for i in range(self.n):
                min = find_the_min_element(k, i, arr2)
                # swap min and ith element of array
                temp = arr2[min]
                arr2[min]= arr2[i]
                arr2[i] =temp

            for i in enumerate(SSSort.arr):
                print(i)


        pid = os.getpid()  # current process
        print("start thread")
        psutil.Process(pid).cpu_percent(interval=0)
        th = threading.Thread(target=selectionSort, args=(k, arr2))
        th.start()
        while th.is_alive():
            print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))
        print("all done")


       


    def main(self):
        self.generate1000_5digit_RandomNum()
        self.cpu_util_info()



if __name__ == "__main__":
    objSSSort = SSSort(20)
    objSSSort.main()

Quote

start thread
CPU usage:98.4
CPU usage:98.4
CPU usage:98.4
CPU usage:100.3
CPU usage:100.3
CPU usage:101.3
CPU usage:100.3
CPU usage:100.3
CPU usage:100.3
CPU usage:99.3
CPU usage:100.3
CPU usage:99.8
CPU usage:99.3
CPU usage:100.4
CPU usage:100.3
CPU usage:101.3
CPU usage:101.3
CPU usage:100.3
CPU usage:100.3
CPU usage:99.8
CPU usage:99.3
CPU usage:99.4
CPU usage:100.3
CPU usage:100.8
CPU usage:101.3
CPU usage:100.3
CPU usage:99.3
CPU usage:100.3
CPU usage:100.3
CPU usage:99.3
CPU usage:100.3
CPU usage:99.8
CPU usage:100.3
CPU usage:101.3
CPU usage:101.3
CPU usage:101.

I have two problems:

1) Multiple out for CPU Utilization statement

2) Sorting is not correct.

Somebody please correct my code. Its showing multiple cpu utlization and sorting is not correct.

 

Zulfi.

Edited March 24, 2020 by zak100

[Ghideon]

7 hours ago, zak100 said:

Do you mean that the blocking call stops the execution of all processes and then returns the cpu time for a particular process?

No. AFAIK the thread making the blocking call is blocked until the call is complete. Which means that the process is blocked if the process is single threaded. It should not affect the state of other running processes in normal* cases.

 

2 hours ago, zak100 said:

I have two problems:

1) Multiple out for CPU Utilization statement

First the CPU issue. It works as intended: psutil.Process(pid).cpu_percent(interval=1)) returns non-zero. The sorting takes more than the intervall so there are more than one report printed. 

2 hours ago, zak100 said:

Somebody please correct my code.

Sorry, not going to happen at this time. You need to explain what you want to achieve. I can post running code, I can explain how it works, I can explain why your code behaves the way it does, how and why it produces some error message or unintended output. But I prefer** not to guess your requirements. One can have one average CPU measurement for the whole sorting or one measurement per second (my guess from your interval=1) or having the sorting complete within a second or something else. Which one to implement depends on what task you need your code to solve.

 

Let's look at the sorting issue

2 hours ago, zak100 said:

2) Sorting is not correct.

 

You create a copy of SSSort.arr and then sort the copy?

arr2 = SSSort.arr[:]

And then prints the unsorted original SSSort.arr? 

for i in enumerate(SSSort.arr):
                print(i)

 

 

*) Details about threading, processes and architectures I consider off topic for this thread
**) Because my opinion is that you will learn from trying to understand what and why you require a certain behaviour. Had this been a professional situation involving time to market, office politics, funding, stakeholders, unclear market demands, risks, compliance and other factors in the reality of system development then my approach regarding unclear requirements could likely be (very) different.

Edited March 24, 2020 by Ghideon
answer to the sorting issue

[zak100]

Hi,

Thanks  a lot for correcting my code. God bless you. 

I want to now focus why I am getting several CPU usage messages.

Quote

One can have one average CPU measurement for the whole sorting or one measurement per second (my guess from your interval=1) or having the sorting complete within a second or something else. Which one to implement depends on what task you need your code to solve.

 

I just want two messages. One message at the start i.e. when I   start the measuring of CPU utilization and one at the end i.e. message stating that the sorting completed and for the entire completed process, the total CPU utilization is this much. This is what you were doing.

Right now my code is working in the following way:

Quote

random 80381
random 88207
random 70287
random 78138
start thread
CPU usage:100.4
CPU usage:99.8
(0, 10028)
(1, 10056)

i.e. it is printing the CPU usage after generating random numbers. This happens when the task is assigned to the thread and "th.start()" is executed and the thread enters the while loop. It might be possible that this line:

print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1)))

 

is being executed by both the main thread and the new thread. Please guide me.

You have pointed out some very good issues. I surely want to focus on them but right now my objective is to make my program behave in the same way as yours is working. But at this point I want to focus on cpu utilization. Kindly guide me.

 

Zulfi.

 

Edited March 25, 2020 by zak100

[Ghideon]

41 minutes ago, zak100 said:

You have pointed out some very good issues. I surely want to focus on them but right now my objective is to make my program behave in the same way as yours is working.

You posted a program that sorts 20 numbers and an output claiming that the sorting is using almost 100% CPU for many seconds. That doesn't sound plausible. Is the output from the program you posted?

Edited March 25, 2020 by Ghideon
spelling

[zak100]

Hi,

Thanks for your interest in solving my problem. Through your guidance, I am able to print cpu_percent. Of course I have to use 10,000 integer numbers. Now the problem is w.r.t printing of cpu_percent twice as I discussed in post#24. 

 

Please guide me why the code is printing cpu_percent() message twice.

May God bless you with great success.

 

Zulfi.