как заменить только \ n, после которого есть символ

Question

Я преобразовал pdf в txt с помощью pdfminer. Проблема в том, что pdfminer добавляет \ n после конца строки в pdf, но предложение на этом не заканчивается. вы можете видеть, что каждая строка воспринимается как предложение в нижеследующем тексте, что неверно. Я также дал другую версию текста, чтобы показать, где находятся символы новых строк. например,

quan-
tum population.

должно быть в одном предложении. Я заменяю \ n на ", и проблема решена. Но другие \ n также заменяют, чего я не хочу.

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)

Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg

Abstract
With advancement in Quantum computing, classical algorithms are adapted and integrated
with Quantum properties such as qubit representation and entanglement. Although these
properties perform better however pre-mature convergence is the main issue in Quantum
Evolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-
tum population. In this paper, we introduced a new way to update the quantum population
of QEA to avoid premature convergence

'Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, 
Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical 
algorithms are adapted and integrated\nwith Quantum properties such as qubit representation 
and entanglement', ' Although these\nproperties perform better however pre-mature 
convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only 
the best individual to update quan-\ntum population', ' In this paper, we introduced a new 
way to update the quantum population\nof QEA to avoid premature convergence',

Я пробовал этот код.

lines =tokenize.sent_tokenize(txt_str)
for l in lines:
    s = l.replace('\n', '')
    print(s)

Это приводит к следующему.

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer BegAbstractWith advancement in Quantum computing, classical algorithms are adapted and integratedwith Quantum properties such as qubit representation and entanglement.
Although theseproperties perform better however pre-mature convergence is the main issue in QuantumEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-tum population.
In this paper, we introduced a new way to update the quantum populationof QEA to avoid premature convergence.

но это нежелательный текст. Я хочу текст в этой версии.

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)

Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg

Abstract
With advancement in Quantum computing, classical algorithms are adapted and integrated with Quantum properties such as qubit representation and entanglement. Although these properties perform better however pre-mature convergence is the main issue in Quantum Evolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-tum population. In this paper, we introduced a new way to update the quantum population of QEA to avoid premature convergence

Я не хочу, чтобы пустые строки исчезали. Надеюсь, ты понял.

Answer 1

(?<=\S)(?<!\bAbstract)\n(?=\S)

Вы можете попробовать это. См. Демонстрацию.

https://regex101.com/r/crj3aD/1

Python скрипт:

inp = "Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical algorithms are adapted and integrated\nwith Quantum properties such as qubit representation and entanglement', ' Although these\nproperties perform better however pre-mature convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-\ntum population', ' In this paper, we introduced a new way to update the quantum population\nof QEA to avoid premature convergence"

output = re.sub(r'(?<=\S)(?<!\bAbstract)\n(?=\S)', ' ', inp)
print(output)

Там - другие условия.

(?<=\S)(?<!\bAbstract)(?:\n|\\n)(?=\S)

Попробуйте это для другого условия.

https://regex101.com/r/crj3aD/2

Answer 2

Я даже не буду пытаться использовать здесь одно регулярное выражение. Требование:

• если перевод строки находится в конце файла, сохранить его
• иначе, если за новой строкой следует еще одна новая строка, оставить ее
• иначе, если строка короткая , оставьте новую строку (короткая, вероятно, меньше 60 или 70)
• иначе, если новой строке предшествует da sh (-), удалите оба символа da sh и новая строка
• иначе замените новую строку пробелом

Это должно быть довольно надежным, но я не могу представить, чтобы выполнить это с помощью регулярных выражений. Предполагая, что ввод:

orig = ['Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical algorithms are adapted and integrated\nwith Quantum properties such as qubit representation and entanglement',
        ' Although these\nproperties perform better however pre-mature convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-\ntum population',
        ' In this paper, we introduced a new way to update the quantum population\nof QEA to avoid premature convergence']

Я бы использовал:

pre = '.'.join(orig).split('\n')
res = []
old = None
for line in pre:
    if old is None:
        old = line
        continue
    if len(line) == 0:
        res.extend([old, '\n'])
    elif len(old) < 70:
        res.extend([old, '\n'])
    elif old[-1] == '-':
        res.append(old[:-1])
    else:
        res.extend([old, ' '])
    old = line


text = ''.join(res)

Он дает для print(text)

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)

Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg

Abstract
With advancement in Quantum computing, classical algorithms are adapted and integrated with Quantum properties such as qubit representation and entanglement. Although these properties perform better however pre-mature convergence is the main issue in Quantum Evolutionary Algorithms(QEA) because QEA uses only the best individual to update quantum population. In this paper, we introduced a new way to update the quantum population 

Answer 3

Чтобы ответить на этот вопрос, каждую строку нужно рассматривать вместе со строкой, которая следует за ней. Правило здесь - удалить символы новой строки, если применимо все следующее:

• это не последняя строка файла
• строка содержит как минимум 2 слова
• следующая строка содержит не менее 1 слова

Проблема удобно решается с помощью функции генератора, которая будет выдавать пары последовательных строк за раз. Это четко отделяет logi c для итерации по файлу от logi c для принятия решения, когда удалять новые строки.

#!/usr/bin/env python

def num_words_line(line):
    return len(line.split())

def iter_lines(input_file):
    """
    yields pairs of adjacent lines
    """
    with open(input_file) as f:
        previous = next(f)
        for line in f:
            yield (previous, line)
            previous = line
        yield (line, None)


def fix_newlines(input_file, output_file):
    with open(output_file, "w") as fout:
        for line, next_line in iter_lines(input_file):
            if (next_line != None and
                num_words_line(line) > 1 and
                num_words_line(next_line) > 0):
                line = line.replace("\n", " ")
            fout.write(line)


if __name__ == '__main__':
    fix_newlines("input.txt", "output.txt")

Дает:

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)

Muhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg

Abstract
With advancement in Quantum computing, classical algorithms are adapted and integrated with Quantum properties such as qubit representation and entanglement. Although these properties perform better however pre-mature convergence is the main issue in Quantum Evolutionary Algorithms(QEA) because QEA uses only the best individual to update quan- tum population. In this paper, we introduced a new way to update the quantum population of QEA to avoid premature convergence

Обратите внимание, число слов в каждой строке рассчитывается дважды. Для немного большей эффективности это можно исправить за счет немного большего количества кода, изменив iter_lines на вызов num_words_line в каждой строке при чтении, а также для получения длин каждой строки в паре. как сами строки. Но тогда было бы менее четкое разделение logi c между iter_lines и fix_newlines.

Для управления строками в памяти вместо чтения и записи файлов можно было бы использовать немного другую версию:

#!/usr/bin/env python

def num_words_line(line):
    return len(line.split())


def iter_lines(input_string):
    """
    yields pairs of adjacent lines
    """
    iterator = iter(input_string.strip().split("\n"))
    previous = next(iterator)
    for line in iterator:
        yield (previous, line)
        previous = line
    yield (line, None)


def fix_newlines(input_string, from_file=True):
    output = ''
    for line, next_line in iter_lines(input_string):
        newline = not (next_line != None and
                       num_words_line(line) > 1 and
                       num_words_line(next_line) > 0)
        output += line
        if newline:
            output += "\n"
        else:
            output += " "
    return output

if __name__ == '__main__':

    input_text = ['Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba, Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical algorithms are adapted and integrated\nwith Quantum properties such as qubit representation and entanglement', ' Although these\nproperties perform better however pre-mature convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only the best individual to update quan-\ntum population', ' In this paper, we introduced a new way to update the quantum population\nof QEA to avoid premature convergence',]
    str = ' '.join(input_text)

    print(fix_newlines(str))

Answer 4

Это должно сделать это за вас:

import re
pattern = re.compile(r"^(.*\(BQCEA\))(.*Beg)(Abstract)(With.*)", re.DOTALL)

try:
    with open('sample.txt', 'r') as f:
        line = f.read()
        # remove some unwanted characters
        r = line.replace('\\n', "").replace("'", "").replace("\n", "")
        print(r)
        for match in re.finditer(pattern, r):
            print(match.group(1))
            print('\n')
            print(match.group(2))
            print('\n')
            print(match.group(3))
            print(match.group(4))
except Exception as er:
    print(er)

Вывод:

Balanced Quantum Classical Evolutionary Algorithm(BQCEA)


Muhammad Shahid, Hasan Mujtaba,Muhammad Asim, Omer Beg


Abstract
With advancement in Quantum computing, classicalalgorithms are adapted and integratedwith Quantum properties such as qubit representationand entanglement,  Although theseproperties perform better however pre-matureconvergence is the main issue in QuantumEvolutionary Algorithms(QEA) because QEA uses onlythe best individual to update quan-tum population,  In this paper, we introduced a newway to update the quantum populationof QEA to avoid premature convergence

Пример:

'Balanced Quantum Classical Evolutionary Algorithm(BQCEA)\n\nMuhammad Shahid, Hasan Mujtaba,
Muhammad Asim, Omer Beg\n\nAbstract\nWith advancement in Quantum computing, classical
algorithms are adapted and integrated\nwith Quantum properties such as qubit representation
and entanglement', ' Although these\nproperties perform better however pre-mature
convergence is the main issue in Quantum\nEvolutionary Algorithms(QEA) because QEA uses only
the best individual to update quan-\ntum population', ' In this paper, we introduced a new
way to update the quantum population\nof QEA to avoid premature convergence'

Answer 5

строк = tokenize.sent_tokenize (txt_str)

s = lines.replace ('\ n', '')

print (s)