zak100 Posted March 21, 2020 Posted March 21, 2020 Hi, I am trying to display system information in Python. I am now getting following error: Quote Traceback (most recent call last): File "/home/zulfi/PycharmProjects/Classes/psutilPrintInfo.py", line 31, in <module> obj_Counting_SysInfo.main() File "/home/zulfi/PycharmProjects/Classes/psutilPrintInfo.py", line 27, in main self.system_info() File "/home/zulfi/PycharmProjects/Classes/psutilPrintInfo.py", line 16, in system_info mem = process.get_memory_info()[0] / float(2 ** 30) # memory in GB AttributeError: 'Process' object has no attribute 'get_memory_info' My program is below: import psutil import os class Counting_SysInfo: def __init__(self, i): self.i = i def Print_Numbers(self): for i in range(50): print (i) def system_info(self): # return the memory usage in MB self.Print_Numbers() process = psutil.Process(os.getpid()) mem = process.get_memory_info()[0] / float(2 ** 30) # memory in GB cpu = process.cpu_percent() / psutil.cpu_count() disk = process.disk_usage(part.mountpoint).percent # https://github.com/giampaolo/psutil/blob/master/scripts/disk_usage.py vMem = int(process.virtual_memory().available) print(process.memory_info().rss) # in bytes RSS = process.memory_info().rss # https://stackoverflow.com/questions/17990227/whats-the-unit-of-rss-in-psutil-process-get-memory-info pageFaults = process.memory_info().pfaults # https://readthedocs.org/projects/giamptest/downloads/pdf/latest/ print("mem= ", mem, "cpu= ", cpu, 'disk= ',disk, "VMem= ", vMem, "RSS= ", RSS, "page faults=", pageFaults ) def main(self): self.system_info() if __name__ == "__main__": obj_Counting_SysInfo = Counting_SysInfo(10) obj_Counting_SysInfo.main() Somebody please guide me. Zulfi.
Strange Posted March 21, 2020 Posted March 21, 2020 You have: 42 minutes ago, zak100 said: mem = process.get_memory_info()... And: 42 minutes ago, zak100 said: pageFaults = process.memory_info()... One of them is wrong ... 1
Ghideon Posted March 21, 2020 Posted March 21, 2020 (edited) 54 minutes ago, zak100 said: Somebody please guide me. 1: Error message says (bold by me): Quote AttributeError: 'Process' object has no attribute 'get_memory_info' So get_memory_info does not exist in a process object. What happens if you replace get_memory_info with an attribute that exists? 2: Reading here: https://psutil.readthedocs.io/en/latest/#psutil.Process.memory_info gives some hints, "memory_info" might be useful. (Or use @Strange's suggestion above) 3: To reduce the risk that unnecessary details we test the change in isolation with three lines of code only. First repeat the error import psutil import os print(psutil.Process(os.getpid()).get_memory_info()[0]) Output: Traceback (most recent call last): File "main.py", line 3, in <module> print(psutil.Process(os.getpid()).get_memory_info()[0]) AttributeError: 'Process' object has no attribute 'get_memory_info' 4: Try a modified version import psutil import os print(psutil.Process(os.getpid()).memory_info()[0]) 144... (example ) 5: Try the whole program with the change above You might want to study more about how to debug code. It may help in the long run to be able to solve the most basic issues without guiding. Edited March 21, 2020 by Ghideon x-post with Strange 1
Strange Posted March 21, 2020 Posted March 21, 2020 12 minutes ago, Ghideon said: 1: Error message says (bold by me): 12 minutes ago, Ghideon said: You might want to study more about how to debug code. It may help in the long run to be able to solve the most basic issues without guiding. Good points. It is always a good idea to think about what the error message says and how it relates to the code you have written. Sometimes error messages are really unhelpful, but usually they give you a good clue to where the problem is. Python error messages can be confusing because they show a traceback through the calls that called the program. Usually, the last line of code referenced ("line 16, in system_info" in this example) is where the problem is (occasionally, it might be that the problem is further back because of how the function was called or the parameters it was passed). And the very last line indicates the nature of the problem. 1
zak100 Posted March 22, 2020 Author Posted March 22, 2020 Hi, Memory part worked, now I am getting error with disk usage: import psutil import os print(psutil.Process(os.getpid()).disk_usage('/').percent) Error is: Quote print(psutil.Process(os.getpid()).disk_usage('/').percent) AttributeError: 'Process' object has no attribute 'disk_usage' I found the following link: https://pypi.org/project/psutil/ Somebody please guide me. Zulfi.
Strange Posted March 22, 2020 Posted March 22, 2020 Disk usage appears to be an attribute of psutil, not pustil.Process: https://psutil.readthedocs.io/en/latest/#psutil.disk_usage 1
Ghideon Posted March 22, 2020 Posted March 22, 2020 (edited) 15 hours ago, Strange said: Python error messages can be confusing because they show a traceback through the calls that called the program. Usually, the last line of code referenced ("line 16, in system_info" in this example) is where the problem is (occasionally, it might be that the problem is further back because of how the function was called or the parameters it was passed). And the very last line indicates the nature of the problem. Good that you pointed that out! (Reading the trace is one of those things I "just do" without thinking). Interpreting the error messages for python is similar to other environments, it is a useful skill to have in your "set of tools" when tracking down errors and bugs. Here is a link that may help https://realpython.com/python-traceback/. 31 minutes ago, zak100 said: Memory part worked, now I am getting error with disk usage: Good to know. 31 minutes ago, zak100 said: Error is: Quote print(psutil.Process(os.getpid()).disk_usage('/').percent) AttributeError: 'Process' object has no attribute 'disk_usage' If you apply what we said regarding the first problem above, do you see what is causing the error? If you check the psutil documentation (for instance at the link you provided) what does it say about disk usage? If you check the reference in your code (https://github.com/giampaolo/psutil/blob/master/scripts/disk_usage.py) how and why is that code different that yours? edit: Please review your requirement; if your program had worked you would probably have got the disk usage for a single process, not the disk usage for a mount point (they may differ) Edited March 22, 2020 by Ghideon x-post with Strange (I need a shortcut for that, happens a lot :-) ) 1
zak100 Posted March 22, 2020 Author Posted March 22, 2020 Hi, print(psutil.disk_usage('/')) I got following answer: Quote sdiskusage(total=982900588544, used=141382160384, free=791518343168, percent=15.2) 12050432 Is the above okay, can I find it for a process? because I have to compare two processes. print(psutil.Process(os.getpid()).cpu_percent(interval=1) But its giving: 0.0 why? Please guide me. For the following two I am getting attribute error: print(psutil.Process(os.getpid()).virtual_memory().available)//no attribute print(psutil.Process(os.getpid()).memory_info().pfaults)//no aatribute Somebody please guide me. Zulfi.
Ghideon Posted March 22, 2020 Posted March 22, 2020 22 minutes ago, zak100 said: Is the above okay, can I find it for a process? because I have to compare two processes. Do you expect the free disk space to be depending on the process? 1 hour ago, zak100 said: But its giving: 0.0 why? Please guide me. If you perform a call to measure the CPU load from the process and the process performs no work then 0.0 seems normal? 1
Ghideon Posted March 22, 2020 Posted March 22, 2020 1 hour ago, zak100 said: print(psutil.Process(os.getpid()).virtual_memory().available)//no attribute print(psutil.Process(os.getpid()).memory_info().pfaults)//no aatribute Same as above. Read about memory_info available in process class. What is your requirement? The process class will get you the virtual memory used by that process. You seem to want to know how much free memory there is left? 1
zak100 Posted March 22, 2020 Author Posted March 22, 2020 Hi, Thanks for your cooperation, I am still getting zero for cpu usage: import psutil import os #import memory_usage print("memory usage=", psutil.Process(os.getpid()).memory_info()[0]) print(psutil.disk_usage('/')) print (psutil.Process(os.getpid()).cpu_percent(interval=1)) print("virtual mem= ",psutil.virtual_memory().available) #print(psutil.Process(os.getpid()).memory_info().pfaults) print("rss=", psutil.Process(os.getpid()).memory_info().rss) print (psutil.Process(os.getpid()).cpu_percent(interval=1)) Please guide me how to get some non-zero for cpu utilization. For page faults, following link says: https://readthedocs.org/projects/giamptest/downloads/pdf/latest/ linix, its shared. I am working on linix. Somebody please guide me about page faults and memory usage. Zulfi.
Ghideon Posted March 22, 2020 Posted March 22, 2020 30 minutes ago, zak100 said: I am still getting zero for cpu usage: If CPU usage is zero then a report of zero cpu is expected? If your process is single threaded and you make a blocking call, how much is the process using the CPU? What is the problem with zero of the CPU usage is zero?* 54 minutes ago, zak100 said: Please guide me how to get some non-zero for cpu utilization. What it is that you want your program to achieve? Do you want to generate CPU load, just to get a non-zero number?* 1 hour ago, zak100 said: Somebody please guide me about page faults and memory usage. Can you be more specific? Does the code give you unexpected output? *) I'll post questions instead of code suggestions at this time. If I just post a program that runs there is a risk that it solves the wrong problem.
Strange Posted March 22, 2020 Posted March 22, 2020 1 hour ago, zak100 said: Please guide me how to get some non-zero for cpu utilization. You would need to get the process to do some significant work to get a non-zero value for this. You had a previous thread about sorting, didn't you. So maybe get the program to sort a large list of numbers before getting the cpu utilisation. 1 hour ago, zak100 said: For page faults, following link says: https://readthedocs.org/projects/giamptest/downloads/pdf/latest/ linix, its shared. I am working on linix. Somebody please guide me about page faults and memory usage. The 'shared' value is the amount of memory that could be shared with other processes. Not the number of page faults. Quote shared: (Linux) memory that could be potentially shared with other processes. This matches “top“‘s SHR column). https://psutil.readthedocs.io/en/latest/#psutil.Process.memory_info It doesn't look as if you can get page faults in Linux. The 'dirty' value might be the closest thing you can get. Note that several of the Linux fields refer to 'top'. You can find a description of those fields in the 'top' man page.
zak100 Posted March 23, 2020 Author Posted March 23, 2020 (edited) Hi and Good Morning, CPU utilization information is still zero. I tried with the sorting code: def systeminfo(self): print("memory usage=", psutil.Process(os.getpid()).memory_info()[0]) print(psutil.disk_usage('/')) print(psutil.Process(os.getpid()).cpu_percent(interval=1)) print("virtual mem= ", psutil.virtual_memory().available) # print(psutil.Process(os.getpid()).memory_info().pfaults) print("rss=", psutil.Process(os.getpid()).memory_info().rss) self.selectionSort() print(psutil.Process(os.getpid()).cpu_percent(interval=1)) def main(self): self.generate1000_5digit_RandomNum() self.systeminfo() if __name__ == "__main__": objSSSort = SSSort(1000) objSSSort.main() I tried sorting with 1000 elements. Quote memory usage= 15269888 sdiskusage(total=982900588544, used=141417086976, free=791483416576, percent=15.2) 0.0 virtual mem= 12050067456 rss= 15269888 (0, 10000) and at other place: Quote (998, 99919) (999, 99927) 0.0 Somebody please guide me. Zulfi. Edited March 23, 2020 by zak100
Strange Posted March 23, 2020 Posted March 23, 2020 6 minutes ago, zak100 said: Hi and Good Morning, CPU utilization information is still zero. I tried with the sorting code: You need to read the documentation for this function: Quote Warning: the first time this method is called with interval = 0.0 or None it will return a meaningless 0.0 value which you are supposed to ignore. https://psutil.readthedocs.io/en/latest/#psutil.Process.cpu_percent
Ghideon Posted March 23, 2020 Posted March 23, 2020 (edited) 26 minutes ago, Strange said: You need to read the documentation Agree. Here is an attempt at describing. Extract from code posted above. self.selectionSort() print(psutil.Process(os.getpid()).cpu_percent(interval=1)) First the sorting is performed. That means there is load on the CPU for a while. Then there is a call to cpu_percent with the interval set to 1. If I got it right from the documentation that means the process waits (blocks) for one second and then reports the CPU usage during that second. Since sorting is already done and the process is not performing work other than the blocking call to cpu_percent then the expected CPU load is zero, correct? I would need to study this more, but your code seem to be intended for a process with multiple threads where other threads are doing work while one thread calls cpu_percent for measurements. (A longer explanation than the short comment I made above) Edited March 23, 2020 by Ghideon
Strange Posted March 23, 2020 Posted March 23, 2020 56 minutes ago, Ghideon said: Agree. But I failed to read the code: I hadn't noticed that the call has "interval=1". So my highlighting that warning was incorrect.
zak100 Posted March 23, 2020 Author Posted March 23, 2020 Hi, Thanks for your interest in my problem. I have not explicitly created any thread. I have used the cpu-percent before and after psutil but I am getting zero in both the cases. Somebody please guide me. Zulfi. Hi, I am trying the selection with 1 million integers but my system hangs up. I am using following following command: if __name__ == "__main__": objSSSort = SSSort(1000000) objSSSort.main() In the above code, I am trying to run my selection sort program with 1000000 integers How to solve this problem in Python? Is there any size limitation with integers in python? Zulfi.
Ghideon Posted March 23, 2020 Posted March 23, 2020 (edited) 2 hours ago, zak100 said: Thanks for your interest in my problem. I have not explicitly created any thread. I have used the cpu-percent before and after psutil but I am getting zero in both the cases. Somebody please guide me. You are still not clear about what you want to do or what you expect to happen. Have you yet read about blocking calls and how they work in this case? Here is an ugly attempt*. I guess that you need cpu_percent(interval=1) to return non-zero for the current process. So I create one thread doing some work while sampling CPU load. import psutil import os import random import threading pid=os.getpid() # current process # define some meaningless task for the CPU def sorting(): print("sorting()") for v in range(100): sequence = [i for i in range(20000)] random.shuffle(sequence) sequence.sort(); print("sorting() done") print("start thread") psutil.Process(pid).cpu_percent(interval=0) th=threading.Thread(target=sorting,args=()) th.start() while th.is_alive(): print("CPU usage:"+ str(psutil.Process(pid).cpu_percent(interval=1))) print("all done") 2 hours ago, zak100 said: my system hangs up. The system did not bother to produce an error message? Out of memory error? 2 hours ago, zak100 said: Is there any size limitation with integers in python? >>>sys.maxint 9223372036854775807 (Which is 2^63 - 1) Not necessarily the same on all versions and on all systems. Here is a link: https://lmgtfy.com/?q=Is+there+any+size+limitation+with+integers+in+python%3F *) have limited time, this is a quick hack that runs, it is not necessarily giving any meaningful measurements. Edited March 23, 2020 by Ghideon 1
zak100 Posted March 24, 2020 Author Posted March 24, 2020 (edited) Hi, Thanks for your program. It looks that the key is creating another thread. I also tried with large values but did not create a separate thread. (interval=0) not important. Quote Have you yet read about blocking calls and how they work in this case? They wait for input. Do you mean that the blocking call stops the execution of all processes and then returns the cpu time for a particular process? Zulfi. Edited March 24, 2020 by zak100
zak100 Posted March 24, 2020 Author Posted March 24, 2020 (edited) Hi, Thanks, your trick succeeded. I have to make changes in the code. I have used inner function. Sorting code got disturbed. Its showing multiple output for cpu utlization. Somebody please correct my code. import random import psutil import os import threading class SSSort: arr = [] # class var access by className.arr def __init__(self, n): self.n = n def generate1000_5digit_RandomNum(self): for i in range(self.n): num = random.randint(10000, 99999) SSSort.arr.append(num) for j in range(self.n): print("random {0}".format(SSSort.arr[j])) def cpu_util_info(self): k= self.n arr2 = SSSort.arr[:] def find_the_min_element(k, i, arr2): min = i for j in range(i+1, k): if (arr2[j] < arr2[min]): min = j return min def selectionSort(k, arr2): for i in range(self.n): min = find_the_min_element(k, i, arr2) # swap min and ith element of array temp = arr2[min] arr2[min]= arr2[i] arr2[i] =temp for i in enumerate(SSSort.arr): print(i) pid = os.getpid() # current process print("start thread") psutil.Process(pid).cpu_percent(interval=0) th = threading.Thread(target=selectionSort, args=(k, arr2)) th.start() while th.is_alive(): print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1))) print("all done") def main(self): self.generate1000_5digit_RandomNum() self.cpu_util_info() if __name__ == "__main__": objSSSort = SSSort(20) objSSSort.main() Quote start thread CPU usage:98.4 CPU usage:98.4 CPU usage:98.4 CPU usage:100.3 CPU usage:100.3 CPU usage:101.3 CPU usage:100.3 CPU usage:100.3 CPU usage:100.3 CPU usage:99.3 CPU usage:100.3 CPU usage:99.8 CPU usage:99.3 CPU usage:100.4 CPU usage:100.3 CPU usage:101.3 CPU usage:101.3 CPU usage:100.3 CPU usage:100.3 CPU usage:99.8 CPU usage:99.3 CPU usage:99.4 CPU usage:100.3 CPU usage:100.8 CPU usage:101.3 CPU usage:100.3 CPU usage:99.3 CPU usage:100.3 CPU usage:100.3 CPU usage:99.3 CPU usage:100.3 CPU usage:99.8 CPU usage:100.3 CPU usage:101.3 CPU usage:101.3 CPU usage:101. I have two problems: 1) Multiple out for CPU Utilization statement 2) Sorting is not correct. Somebody please correct my code. Its showing multiple cpu utlization and sorting is not correct. Zulfi. Edited March 24, 2020 by zak100
Ghideon Posted March 24, 2020 Posted March 24, 2020 (edited) 7 hours ago, zak100 said: Do you mean that the blocking call stops the execution of all processes and then returns the cpu time for a particular process? No. AFAIK the thread making the blocking call is blocked until the call is complete. Which means that the process is blocked if the process is single threaded. It should not affect the state of other running processes in normal* cases. 2 hours ago, zak100 said: I have two problems: 1) Multiple out for CPU Utilization statement First the CPU issue. It works as intended: psutil.Process(pid).cpu_percent(interval=1)) returns non-zero. The sorting takes more than the intervall so there are more than one report printed. 2 hours ago, zak100 said: Somebody please correct my code. Sorry, not going to happen at this time. You need to explain what you want to achieve. I can post running code, I can explain how it works, I can explain why your code behaves the way it does, how and why it produces some error message or unintended output. But I prefer** not to guess your requirements. One can have one average CPU measurement for the whole sorting or one measurement per second (my guess from your interval=1) or having the sorting complete within a second or something else. Which one to implement depends on what task you need your code to solve. Let's look at the sorting issue 2 hours ago, zak100 said: 2) Sorting is not correct. You create a copy of SSSort.arr and then sort the copy? arr2 = SSSort.arr[:] And then prints the unsorted original SSSort.arr? for i in enumerate(SSSort.arr): print(i) *) Details about threading, processes and architectures I consider off topic for this thread **) Because my opinion is that you will learn from trying to understand what and why you require a certain behaviour. Had this been a professional situation involving time to market, office politics, funding, stakeholders, unclear market demands, risks, compliance and other factors in the reality of system development then my approach regarding unclear requirements could likely be (very) different. Edited March 24, 2020 by Ghideon answer to the sorting issue 1
zak100 Posted March 25, 2020 Author Posted March 25, 2020 (edited) Hi, Thanks a lot for correcting my code. God bless you. I want to now focus why I am getting several CPU usage messages. Quote One can have one average CPU measurement for the whole sorting or one measurement per second (my guess from your interval=1) or having the sorting complete within a second or something else. Which one to implement depends on what task you need your code to solve. I just want two messages. One message at the start i.e. when I start the measuring of CPU utilization and one at the end i.e. message stating that the sorting completed and for the entire completed process, the total CPU utilization is this much. This is what you were doing. Right now my code is working in the following way: Quote random 80381 random 88207 random 70287 random 78138 start thread CPU usage:100.4 CPU usage:99.8 (0, 10028) (1, 10056) i.e. it is printing the CPU usage after generating random numbers. This happens when the task is assigned to the thread and "th.start()" is executed and the thread enters the while loop. It might be possible that this line: print("CPU usage:" + str(psutil.Process(pid).cpu_percent(interval=1))) is being executed by both the main thread and the new thread. Please guide me. You have pointed out some very good issues. I surely want to focus on them but right now my objective is to make my program behave in the same way as yours is working. But at this point I want to focus on cpu utilization. Kindly guide me. Zulfi. Edited March 25, 2020 by zak100
Ghideon Posted March 25, 2020 Posted March 25, 2020 (edited) 41 minutes ago, zak100 said: You have pointed out some very good issues. I surely want to focus on them but right now my objective is to make my program behave in the same way as yours is working. You posted a program that sorts 20 numbers and an output claiming that the sorting is using almost 100% CPU for many seconds. That doesn't sound plausible. Is the output from the program you posted? Edited March 25, 2020 by Ghideon spelling
zak100 Posted March 25, 2020 Author Posted March 25, 2020 Hi, Thanks for your interest in solving my problem. Through your guidance, I am able to print cpu_percent. Of course I have to use 10,000 integer numbers. Now the problem is w.r.t printing of cpu_percent twice as I discussed in post#24. Please guide me why the code is printing cpu_percent() message twice. May God bless you with great success. Zulfi.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now