VideoOpinionMining Documentation

class VideoOpinionMining.vem.EmotionMapGenerator(segments_with_emotion, graph=None)

Bases: object

Class responsible for generating the heatmaps visualization of the classifications.

Args:

segmented_with_emotion (dataframe): sentences to be analyzed with the audio and transcript classification already been done.

graph (networkx graph): graph generated by the multimodal model

Return:

Nothing.

generate_emotion_map(modality, segment_block_size=5)

Generate an emotion wheel for each block of segments in the format of mp4 and png.

Args:

modality (string): for which modality is the heatmap to be created (the opinion excration of the respective modality have to be done befor)

segment_block_size (int): how many phrases are shown in the same frame of the heatmap.

Return:

Nothing.

class VideoOpinionMining.vem.MultimodalOpinionExtractor(segmented_with_emotion)

Bases: object

Class responsible for applying the multimodal model to all the segments of the video after the audio and transcript classification have already been done

Args:

segmented_with_emotion (dataframe): sentences to be analyzed with the audio and transcript classification already been done.

Return:

Nothing.

extract_multimodal_opinions()

Extract multimodal opinions from the segmented video segments with already identified emotion. Add multimodal emotion to each segment using the opinion extraction model.

class VideoOpinionMining.vem.OpinionExtractionModel

Bases: object

Class to declare and store the models utilized for the transcript, audio and video classification

set_model(modality, encoder_text=None, emot_pipe=None)

Start the model to be utilized in each modality. In order to modify the video and audio, manual changes have to be made to this code, while the transcript model can be easily changed passing a pipeline and an encoder

Args:

modality (str): modality that the model will analyze.

emot_pipe (pipeline): pass a pipeline in order to change the model being used to classify the emotions.

encoder_text (encoder): pass the encoder used in the pipeline passed.

Return:

Dataframe with all the segments.

class VideoOpinionMining.vem.OpinionExtractor(segmenter_result, opinion_model)

Bases: object

Class responsible for applying the models to all the segments of the video

Args:

segmenter_result (dataframe): sentences to be analyzed.

opinion_model (OpinionExtractionModel): Model responsible for the analysis.

Return:

Nothing.

extract_opinions()

Extract opinions from the segmented video segments. Add identified emotion to each segment using the opinion extraction model.

class VideoOpinionMining.vem.VEMProcessor

Bases: object

Here is the main class that makes this module, VEMProcessor

process_video(video_file, segment_block_size=10)

Use this to run the models for all the modalities(transcript, audio, video and multimodal) and generate the heatmaps.

Args:

video_file (mp4): video to be analized.

segment_block_size (int): size of the block used in each frame of the heatmap.

Return:

Nothing.

In case you also want the dataframes generated with all the emotions, object_of_class.segmented_video contains it.

class VideoOpinionMining.vem.VideoSegmenter

Bases: object

Class responsible for extracting the transcription from the video, segmenting it in phrases with the timestamps and name of the parts contained in a dataframe

segment_video(video_file)

Receives an MP4 file and returns a list of video segments. Each segment is represented as a series in a dataframe: (start_time, end_time, transcript_text, segment_file.mp4)

Args:

video_file (str): video to be analyzed.

Return:

Dataframe with all the segments.

VideoOpinionMining.vem.get_labels(dataframe, modality='all')