I'm new with Python but i'm a Powershell user so maybe what i'm trying to do is not possible the same way in Python
In Python 3 to learn i'm trying to make a list of the files in a directory and store it into a indexstore variable.
To do that this is what i done :
i created 2 objects Index and Indexstore
class Index(object):
def __init__(self, filepath, size):
self.filepath = filepath
self.size = size
and
class IndexStore(object):
def __init__(self, filepath, size):
self.filepath = filepath
self.size = size
after that i get my filelist from a location on my HDD
listOfFile = os.listdir(SourcePath)
With with list i'm starting a loop where i get the fullpath and the size of the file ( Like ForEach ) in 2 variables fullPath and fileSize
fullPath = os.path.join(SourcePath, entry)
fileSize: int = os.path.getsize(fullPath)
With the values i set the Index Object
setattr(Index, 'filepath', fullPath)
setattr(Index, 'size', fileSize)
And it's working with
pprint(vars(Index))
i have the result
mappingproxy({'dict': <attribute 'dict' of 'Index' objects>,
'doc': None,
'init': <function Index.init at 0x00000271B9C7E940>,
'module': 'main',
'repr': <property object at 0x00000271B9F30590>,
'weakref': <attribute 'weakref' of 'Index' objects>,
'filepath': 'D:\AmigaForever\AmigaOS3.9.ISO',
'size': 28862259})
After that is my problem ! In Powershell if i want to add a object2 to my objectlist1 i just do Objectlist1 += object2 and the work is done but in Python 3.x i tried many things on forums without success best way seems to be :
IndexStore = []
IndexStore.append(Index(fullPath, fileSize))
But the variable Indexstore stay Empty and if i try to print it
print(IndexStore)
pprint(vars(IndexStore))
the run console say :
print(IndexStore)
TypeError: 'tuple' object is not callable
Can you help me please ? Do i'm checking the value of my Indexstore well ?
Or my error is how i'm appending the values ?
I want in a second loop use again the values of the Object array to continue my code.
With the goal of 'Using Python 3 to make a list of the files in a directory and store it into a indexstore variable'.
The first problem I see is that you create a class Indexstore but later completely obviate the class when you assign the variable Indexstore = [].
so given you have a valid list of files from:
listOfFile = os.listdir(SourcePath)
This is an approach that will work:
First build an IndexItem class:
class IndexItem:
def __init__(self, filepath, size):
self.filepath = filepath
self.size = size
def __repr__(self):
# Returns a string representation of the IindexItem
return f"({self.filepath}, {self.size})"
This class has an intialization method which serves to preserve the instantiation values passed during initialization and a repr method which is used to convert the index values into readable text.
Next we create the IndexStore Class as follows:
class IndexStore:
def __init__(self):
self._index = []
def append(self, o: object):
# Append an entry onto the index
self._index.append(o)
def __repr__(self):
# Returns a string representation of self._index
return ', '.join(list(str(x) for x in self._index))
This class includes an instantiation which creates a list to hold the indexItems passed to it, and append method to add IndexItems to the IndexStore and finally a repr to create a readable string of the values.
Finally, we implement the basic functionality required to build the IndexStore as follows:
listOfFile = os.listdir(sourcePath)
index = IndexStore()
for f in listOfFile[:5]:
# For each entry f in listOfFile
fullpath = os.path.join(sourcePath, f)
# add an instantiation of IndexItem to IndexStore
index.append(IndexItem(fullpath, int(os.path.getsize(fullpath))))
print(index)
A simpler and more direct approach to this problem makes use of inherent Python bult-in data structures and capabilities is as follows:
IndexStore = []
listOfFile = os.listdir(sourcePath)
for f in listOfFile[:5]:
# For each entry f in listOfFile
fullpath = os.path.join(sourcePath, f)
# add an instantiation of IndexItem to IndexStore
IndexStore.append((fullpath, int(os.path.getsize(fullpath)))
print(IndexStore)
In this approach, the class definitions are eliminated, and the IndexStore contains a list of tuples with each tuple containing the fullpath to the file and it's size
Related
In ruby I have created a class and array, to read the contents from a text file then output them.
class Album
attr_accessor :artist, :title, :genre, :tracks
end
album = Album.new(album_title, album_artist, album_genre, tracks)
-> tracks is an array of multiple lines read from the text file using a while loop. Context below, a_file/music_file File.new("album.txt", "r")
class Track
attr_accessor :name, :location
def read_track(a_file)
track_title = a_file.gets()
track_location = a_file.gets()
track = Track.new(track_title, track_location)
end
def read_tracks(music_file)
tracks = Array.new()
count = music_file.gets().to_i()
track = music_file
index = 0
while (index < count)
track = read_track(music_file)
tracks << track
index += 1
end
return tracks
end
end
after album = Album.new(album_title, album_artist, album_genre, tracks), I passed the album to a different procedure print_tracks(album), and in print_tracks(album), I have puts album.tracks.
But instead of printing out several lines of track names and track locations, I get something that looks like this:
#<Track:0x000055c028027b08>
#<Track:0x000055c0280277c0>
#<Track:0x000055c028027630>
How do I print out the actual words on the file?
What you are getting in return are instances of your Track class. Each of those instances has access to attributes like name and location as specified under the class definition in attr_accessor. You can change your last statement (return tracks) (take note that return here is not needed since its the last statement in the method, the last thing will be returned by default in ruby).
Try this instead of return tracks
tracks.map{ |track| {track_name: track.name, track_location: track.location} }
This was you will end up with array of hashes with keys of track_name and track_location each containing a value of one track. I am not sure what kind of format you want to return, but this is a rather simple, yet flexible. The most simplistic way would be array of array, which you can get using:
racks.map{ |track| [track.name, track.location] }
You're observing the default behavior defined in Object#to_s and Object#inspect.
Ruby uses the to_s method to convert objects to strings and the inspect method to obtain string representations of objects suitable for debugging. By default, to_s and inspect are more or less the same thing. The only difference is inspect will also show the instance variables.
You can and should override these methods in your Track class. For example:
class Track
def to_s
"#{self.class.name} #{name} # {location}"
end
def inspect
"#<#{to_s}>"
end
end
track.to_s
puts track
# Track: name # /music/name.flac
track.inspect
p track
# #<Track: name # /music/name.flac>
I am new to pytorch. I am trying to create a DataLoader for a dataset of images where each image got a corresponding ground truth (same name):
root:
--->RGB:
------>img1.png
------>img2.png
------>...
------>imgN.png
--->GT:
------>img1.png
------>img2.png
------>...
------>imgN.png
When I use the path for root folder (that contains RGB and GT folders) as input for the torchvision.datasets.ImageFolder it reads all of the images as if they were all intended for input (classified as RGB and GT), and it seems like there is no way to pair the RGB-GT images. I would like to pair the RGB-GT images, shuffle, and divide it to batches of defined size. How can it be done? Any advice will be appreciated.
Thanks.
I think, the good starting point is to use VisionDataset class as a base. What we are going to use here is: DatasetFolder source code. So, we going to create smth similar. You can notice this class depends on two other functions from datasets.folder module: default_loader and make_dataset.
We are not going to modify default_loader, because it's already fine, it just helps us to load images, so we will import it.
But we need a new make_dataset function, that prepared the right pairs of images from root folder. Since original make_dataset pairs images (image paths if to be more precisely) and their root folder as target class (class index) and we have a list of (path, class_to_idx[target]) pairs, but we need (rgb_path, gt_path). Here is the code for new make_dataset:
def make_dataset(root: str) -> list:
"""Reads a directory with data.
Returns a dataset as a list of tuples of paired image paths: (rgb_path, gt_path)
"""
dataset = []
# Our dir names
rgb_dir = 'RGB'
gt_dir = 'GT'
# Get all the filenames from RGB folder
rgb_fnames = sorted(os.listdir(os.path.join(root, rgb_dir)))
# Compare file names from GT folder to file names from RGB:
for gt_fname in sorted(os.listdir(os.path.join(root, gt_dir))):
if gt_fname in rgb_fnames:
# if we have a match - create pair of full path to the corresponding images
rgb_path = os.path.join(root, rgb_dir, gt_fname)
gt_path = os.path.join(root, gt_dir, gt_fname)
item = (rgb_path, gt_path)
# append to the list dataset
dataset.append(item)
else:
continue
return dataset
What do we have now? Let's compare our function with original one:
from torchvision.datasets.folder import make_dataset as make_dataset_original
dataset_original = make_dataset_original(root, {'RGB': 0, 'GT': 1}, extensions='png')
dataset = make_dataset(root)
print('Original make_dataset:')
print(*dataset_original, sep='\n')
print('Our make_dataset:')
print(*dataset, sep='\n')
Original make_dataset:
('./data/GT/img1.png', 1)
('./data/GT/img2.png', 1)
...
('./data/RGB/img1.png', 0)
('./data/RGB/img2.png', 0)
...
Our make_dataset:
('./data/RGB/img1.png', './data/GT/img1.png')
('./data/RGB/img2.png', './data/GT/img2.png')
...
I think it works great) It's time to create our class Dataset. The most important part here is __getitem__ methods, because it imports images, applies transformation and returns a tensors, that can be used by dataloaders. We need to read a pair of images (rgb and gt) and return a tuple of 2 tensor images:
from torchvision.datasets.folder import default_loader
from torchvision.datasets.vision import VisionDataset
class CustomVisionDataset(VisionDataset):
def __init__(self,
root,
loader=default_loader,
rgb_transform=None,
gt_transform=None):
super().__init__(root,
transform=rgb_transform,
target_transform=gt_transform)
# Prepare dataset
samples = make_dataset(self.root)
self.loader = loader
self.samples = samples
# list of RGB images
self.rgb_samples = [s[1] for s in samples]
# list of GT images
self.gt_samples = [s[1] for s in samples]
def __getitem__(self, index):
"""Returns a data sample from our dataset.
"""
# getting our paths to images
rgb_path, gt_path = self.samples[index]
# import each image using loader (by default it's PIL)
rgb_sample = self.loader(rgb_path)
gt_sample = self.loader(gt_path)
# here goes tranforms if needed
# maybe we need different tranforms for each type of image
if self.transform is not None:
rgb_sample = self.transform(rgb_sample)
if self.target_transform is not None:
gt_sample = self.target_transform(gt_sample)
# now we return the right imported pair of images (tensors)
return rgb_sample, gt_sample
def __len__(self):
return len(self.samples)
Let's test it:
from torch.utils.data import DataLoader
from torchvision.transforms import ToTensor
import matplotlib.pyplot as plt
bs=4 # batch size
transforms = ToTensor() # we need this to convert PIL images to Tensor
shuffle = True
dataset = CustomVisionDataset('./data', rgb_transform=transforms, gt_transform=transforms)
dataloader = DataLoader(dataset, batch_size=bs, shuffle=shuffle)
for i, (rgb, gt) in enumerate(dataloader):
print(f'batch {i+1}:')
# some plots
for i in range(bs):
plt.figure(figsize=(10, 5))
plt.subplot(221)
plt.imshow(rgb[i].squeeze().permute(1, 2, 0))
plt.title(f'RGB img{i+1}')
plt.subplot(222)
plt.imshow(gt[i].squeeze().permute(1, 2, 0))
plt.title(f'GT img{i+1}')
plt.show()
Out:
batch 1:
...
Here you can find a notebook with code and simple dummy dataset.
I have some trouble getting my static class to work. There is something I am missing about the scope of functions within a class. If called the script gives me following error:
NameError: global name 'disableCostumFrames' is not defined #
import maya.cmds as cmds
from functools import partial
class Blast:
def createWindow():
# Todo:
# hanldes the gui for the user
windowID = 'window'
if cmds.window(windowID, exists = True):
cmds.deleteUI('window')
window = cmds.window(windowID, title="Blast", iconName='Blast', widthHeight=(400, 200) )
cmds.frameLayout( label='')
cmds.rowColumnLayout( numberOfColumns=4, columnWidth=[(1, 100),(3, 100)] )
cmds.text( label='Start: ' )
global Blast_startFrame
Blast_startFrame = cmds.textField( enable = False)
cmds.text( label=' End: ' )
global Blast_endFrame
Blast_endFrame = cmds.textField( enable = False)
cmds.setParent('..')
cmds.rowColumnLayout( numberOfColumns=2, columnWidth=[(1, 100), (2, 100)] )
cmds.radioCollection()
#cmds.radioButton( label='Full', select = True, onCommand= partial(disableCostumFrames, Blast_startFrame, Blast_endFrame ) )
#cmds.radioButton( label='Costum', onCommand= partial(enableCostumFrames, Blast_startFrame, Blast_endFrame ) )
cmds.setParent('..')
cmds.rowColumnLayout( numberOfColumns=1, columnWidth=[(1, 400), (2, 100)] )
cmds.button( label='Playblast' ,command= 'createPlayblast')
cmds.setParent('..')
cmds.showWindow( window )
return Blast_startFrame, Blast_endFrame
def main():
createWindow()
def enableCostumFrames(Blast_startFrame, Blast_endFrame, *args):
cmds.textField(Blast_startFrame, edit=True, enable=True)
cmds.textField(Blast_endFrame, edit=True, enable=True)
def disableCostumFrames(Blast_startFrame, Blast_endFrame, *args):
cmds.textField(Blast_startFrame, edit=True, text="", enable=False)
cmds.textField(Blast_endFrame, edit=True, text="", enable=False)
How do I need to define these functions within the class? I am calling the module like that:
import sys
Dir = 'c:/Blast'
if Dir not in sys.path:
sys.path.append(Dir)
try: reload(Blast_v011)
except: import Blast_v011
Blast_v011.Blast()
maybe iam doing something wrong on this side? Appreciated any help.
In this case you need to add a self reference to all of the methods in your class. The usual python class looks like this:
class MyClass(object):
def __init__(self):
self.variable = 123
def some_method(self):
print "my variable = ", self.variable
def some_other_method(self):
if self.variable > 1:
self.some_method()
The self reference in the member functions is how you get at class member variables and other functions -- it's python's way of referring to what other languages call this.
Instance methods can only be called on an instance (it's the instance that gets passed in as self). You can make a method that is called on the class itself -- rather than any particular instance of the class -- using the #classmethod decorator. Classmethods also take an argument, but instead of self it's a reference to the class. You use it the same way to get variables defined at the class level, which are shared by all copies of the class:
class HasShared(object):
shared = 99
#classmethod
def a_class_method(cls):
print cls.shared
(You can mix and match class and instance methods in the same class).
You can also make static methods using the #staticmethod decorator. These don't get a default argument at all:
class NotPythonic(object):
#staticmethod
def needs_no_args():
print "call me as NotPythonic.needs_no_args()"
In Python we tend to avoid this formula, since you can get a static method by just creating a function in a module without making a class to hold them. For the example code you posted, I'd probably just make a conventional class using instance methods, since your functions need the names of the gui widgets to be able to actually ask them questions.
I have have written this code defining a class
class OrderRecord:
"""Defines an OrderRecord class, suitable for use in keeping track of order records"""
import tools2
def __init__(self, string):
"""Creates a new OrderRecord object"""
string = string.split(',')
self.date = string[0]
self.location = string[1]
self.name = string[2]
self.colour = string[3]
self.order_num = string[4]
self.cost = 0
def cost_of_order(self):
"""Creates a list of the name and adds up the cost of each letter"""
letter = list(self.name)
for let in letter:
self.cost = self.cost + self.tools2.letter_price(let, self.colour)
return self.cost
def __str__(self):
"""Calls the cost_of_order function and returns the split string in the required format"""
self.cost = self.cost_of_order()
return("Date: {0}\nLocation: {1}\nName: {2}\nColour: \
{3}\nOrder Num: {4}\nCost: {5:.2f}".format(self.date, self.location, \
self.name, self.colour, self.order_num, self.cost))
Now I need to write a function that reads a file containing the following:
20130902,Te Rakipaewhenua,Vinas,parauri,8638
20130909,Te Papaioea,McClary,kikorangi,11643
20131215,Kapiti,Labrie,kikorangi,65291
20141106,Waihopai,Labrie,ma,57910
and returns a dictionary that has the location as the key and lists of OrderRecords as the values.
I know this isn't too hard of a task but I have been stuck on this for awhile because I can't get my head around what to do for it.
Any help would be appreciated.
Maybe something like this. It is not the solution but it has what you need with some modifications.
import collections
dct_result = collections.defaultdict(list)
for line in open('file_path'):
fields = line.split(',')
# index 1 being the second column
dct_result[field(1)].append(OrderRecord( some args ))
I was going to CSV based export/import for large data with app engine. My idea was just simple.
First column of CSV would be key of entity.
If it's not empty, that row means existing entity and should overwrite old one.
Else, that row is new entity and should create new one.
I could export key of entity by adding key property.
class FrontExporter(bulkloader.Exporter):
def __init__(self):
bulkloader.Exporter.__init__(self, 'Front', [
('__key__', str, None),
('name', str, None),
])
But when I was trying to upload CSV, it had failed because bulkloader.Loader.generate_key() was just for "key_name" not "key" itself. That means all exported entities in CSV should have unique 'key_name' if I want to modify-and-reupload them.
class FrontLoader(bulkloader.Loader):
def __init__(self):
bulkloader.Loader.__init__(self, 'Front', [
('_UNUSED', lambda x: None),
('name', lambda x: x.decode('utf-8')),
])
def generate_key(self,i,values):
# first column is key
keystr = values[0]
if len(keystr)==0:
return None
return keystr
I also tried to load key directly without using generate_key(), but both failed.
class FrontLoader(bulkloader.Loader):
def __init__(self):
bulkloader.Loader.__init__(self, 'Front', [
('Key', db.Key), # not working. just create new one.
('__key__', db.Key), # same...
So, how can I overwrite existing entity which has no 'key_name'? It would be horrible if I should give unique name to all entities.....
From the first answer, I could handle this problem. :)
def create_entity(self, values, key_name=None, parent=None):
# if key_name is None:
# print 'key_name is None'
# else:
# print 'key_name=<',key_name,'> : length=',len(key_name)
Validate(values, (list, tuple))
assert len(values) == len(self._Loader__properties), (
'Expected %d columns, found %d.' %
(len(self._Loader__properties), len(values)))
model_class = GetImplementationClass(self.kind)
properties = {
'key_name': key_name,
'parent': parent,
}
for (name, converter), val in zip(self._Loader__properties, values):
if converter is bool and val.lower() in ('0', 'false', 'no'):
val = False
properties[name] = converter(val)
if key_name is None:
entity = model_class(**properties)
#print 'create new one'
else:
entity = model_class.get(key_name)
for key, value in properties.items():
setattr(entity, key, value)
#print 'overwrite old one'
entities = self.handle_entity(entity)
if entities:
if not isinstance(entities, (list, tuple)):
entities = [entities]
for entity in entities:
if not isinstance(entity, db.Model):
raise TypeError('Expected a db.Model, received %s (a %s).' %
(entity, entity.__class__))
return entities
def generate_key(self,i,values):
# first column is key
if values[0] is None or values[0] in ('',' ','-','.'):
return None
return values[0]
Your best option is probably to override create_entity. You'll need to copy most of the existing code there, but modify the constructor to supply a key argument instead of a key_name argument.