See PR #651
This commit restores the functionality that prevents spell checking a
word that is being actively typed at the end of a paragraph.
The goals for the spell check word match regexp are:
A. Words should include those with an apostrophe
*E.g., can't*
B. Words should exclude underscore
*E.g., hello_world is two words*
C. Words in other languages should be recognized
*E.g., French word familiarisé*
D. Spell check should include word at absolute end of line with no
trailing space or punctuation
*E.g., tezt*
E. Spell check should ignore partial words in progress (user typing)
*E.g., paragr while midway through typing paragraph*
This commit addresses all five of the above goals.
HISTORY:
- See issue #166 and commit 6ec0c19 in the 0.5.0 release.
- See issue #283 and commit 63b471e in the 0.7.0 release.
Also fix minor incorrect utf-8 encoding at top of source file.
See issue #611 and pull request #642.
The warning previously added in PR #612 for Manuskript users with Qt
5.11 / 5.12 has shown itself, in my opinion, to be overly annoying.
This is because the warning *always* displays on systems with the
affected Qt versions even though the crash is verified to happen with
the import feature only.
Additionally the process to upgrade to a newer version of PyQt / Qt is
not trivial for users who rely on pre-built packages and do not run
from source code.
Because the crash has been verified with the Import feature only, limit
the scope of the warning to the Import feature.
The user can configure a language for Manuskript in the dialog, but
before that setting is ever written to disk, there is the default
behaviour that tries to auto-detect the best language to show based on
the configuration of the device it is running on.
While doing my due diligence on issue #619, I realized we were relying
on the system locale, which is not necessarily equivalent to the
language the user is working with. Worse still: a user can have multiple
preferred languages for their user interface, and our old approach might
actually offer them the 'wrong' language. This patch fixes this.
It also refactors and comments things a little bit where necessary.
Issue #619 revealed an unintentional overlap between the auto-detection
of the locale and the implicit builtin English language, which resulted
in users being unable to select the builtin English language when their
device was configured with a locale that we happen to have a translation
for. The code has been rewritten to more clearly separate auto-detection
and the final fallback that is the builtin English translations.
This code change sets:
- Fequency Analyzer tool default first tab of "Word frequency"
(was Phrase frequency)
Steps to set default window tab:
1. Start Qt Designer
2. Open .ui file
3. Ensure that each selected window tab is the one desired as default
4. Save .ui file
5. Exit Qt Designer
6. Generate .py file with: make ui
See PR #623
This code change sets:
- Character pane default first tab of "Basic info" (was Notes)
- Character pane Basic info "Name" as the default first field (was Motivation)
- Plots pane default first tab of "Basic info" (was Resolution steps)
Steps to edit tab order and default window tab:
1. Start Qt Designer
2. Open .ui file
3. Choose menu **Edit -> Edit Tab Order**
4. Ctrl-click on item just before the first incorrect tab order item
5. Click other items in order until remaining order is correct
6. Ensure that each selected window tab is the one desired as default
7. Save .ui file
8. Exit Qt Designer
9. Generate .py file with: make ui
See https://doc.qt.io/qt-5/designer-tab-order.html
This code change implements a portion of issue #244
Last time I touched this code, I went in looking for a specific problem,
and came out with a fix specific to that issue. That fix was not wrong,
yet it hardly covered all the problems present in the code once one took
into account issues like:
- local vs remote resources,
- relative vs absolute paths,
- different operating systems behaving differently, and
- Qt being uniquely buggy on different platforms.
The major part of it was fixed by using QUrl.fromUserInput(), which does
the exact kind of auto-detection for the nature of the resource that we
were in need of.
The rest of the issues were fixed by creating a number of test cases and
fixing problems as they popped up. Testing was done in Windows & Ubunty
against the above-mentioned test cases, which can be found in PR #629.
Regarding ImageTooltip.supportedSchemes
When QUrl.fromUserInput() misidentifies the scheme on Linux, it causes
all resemblance between the original request and the reply.request() in
the finished() signal to be lost, which results in this item getting
stuck in the ImageTooltip processing pipeline.
Limiting the supported schemes to the ones most commonly encountered
('file', 'http', 'https' and the schema-less local paths) is the only
reliable method I have found to work around this particular bug in Qt.
Windows 10 has supported a 'dark theme' option for a while, and the fact
Manuskript is like looking into a bastion of bright white while using it
is bothersome to say the least.
Since this is a setting defined as the OS level, I believe this should
be something Manuskript automatically adjusts itself to match, thus the
lack of a configurable setting on the Manuskript end.
Qt has garbage documentation. I dug into Qt 5.0 source code to make sure
the crashing call can be safely left out for older versions, and that is
indeed the case. setSupportedSchemes() appears to be introduced together
with the future to select non-local files in the dialog, but we only
cared about local files to begin with.
I happened to notice the UI briefly locking up when testing my previous
changes on some of the Pandoc exporters. While the bigger mess I found
along the way is more than a little fix can handle, this stopgap measure
will at least stop us from running pandoc when it isn't needed.
According to issue #608 we were silently overwriting files when there
was a suffix being generated for a chosen export filename when one was
missing to begin with. Unfortunately, this helpful feature avoids all
the conveniences offered by QFileDialog in regards to alerting the user
to overwriting an existing file. Worse still, this feature already
exists in QFileDialog and the native APIs it can rely on.
This patch reimplements QFileDialog.getSaveFileName to allow the use of
the default suffix feature as functions.getSaveFileNameWithSuffix and
removes most of the magic involved with the old solution.
It is the only FileDialog in the entire codebase that does not conform
to the rest of the OS like its brethren, and it stuck out like a sore
thumb because of it.
Some bugs are out of our reach to fix, but can still impact the user
considerably. Because losing progress always hurts, we want to make
the user aware of the risks before any tears are shed. (PR #612)
See PR #615
This rename is being done to clarify that when a user enables "Save on
project close" then the project will be saved whenever the user
chooses to close the project or to quit Manuskript.
Note that the actual name of the setting saveOnQuit should also be
changed but instead has been marked as a future TODO because it
involves a change in the project file format.
See PR #615
The Travis CI tests began failing after merging Pull Request #583.
Log snippet:
----------------------------------------------------------------------
...
Ref not implemented
PASSED
manuskript/tests/ui/test_welcome.py::test_autoLoad QXcbConnection: XCB error: 8 (BadMatch), sequence: 613, resource id: 2097162, major code: 42 (SetInputFocus), minor code: 0
QXcbConnection: XCB error: 8 (BadMatch), sequence: 619, resource id: 2097168, major code: 42 (SetInputFocus), minor code: 0
QXcbConnection: XCB error: 8 (BadMatch), sequence: 625, resource id: 2097171, major code: 42 (SetInputFocus), minor code: 0
----------------------------------------------------------------------
When running "pytest -vs" locally, which is a command used in our
.travis.yml file, a dialog to "Save project?" is displayed. Because
the test scripts use the "saveOnQuit" default setting of *enabled*,
the "Save project?" dialog should not be displayed.
In other words when a call is made to close the project, a "Save
project?" dialog is incorrectly displayed because the dirtyProject
flag is set, but so too is saveOnQuit set to True. What should happen
is an automatic save with no prompt. This PR fixes this logic so that
the Travis CI test suite completes successfully.
Turning off the timer for saveTimerNoChanges just like the code
already did in closeProject() for saveTimer fixes this bug. Easy.
But how to prevent this kind of race condition in the future?
Several related routines have been adjusted to fail gracefully or report
a bug to the console when something goes wrong, depending on what is
most suitable for that bit of logic.
Intending to learn more about the way Manuskript goes about saving the
project in order to figure out how to tackle some recent saving-related
issues, I stumbled into learning that Manuskript likes to save data a
whole lot. Too much, in fact. When I close the project with unsaved
changes, I expected those changes to not be saved... but they were. This
completely subverts my expectations of a program using typical
file-based operations involving opening, saving and closing files.
There are three more settings that influence when the program saves, and
I personally consider them a bit overkill or even detrimental to the
stated purpose. What if Manuskript forces a save when nothing was
changed and something goes wrong? Saving too much can in fact be
dangerous!
For now, I have left existing functionality as-is, but I would prefer to
respect the dirty flag I have introduced in this commit for at least the
'save-on-quit' and 'save every X minutes' features. (The third is
smarter and only triggers after noticing changes, so it is less
important.)
Making sure the dirty flag works as expected is the first step in making
such changes in the future.
UI-wise, this commit now offers the user the opportunity to save their
changes, discard them, or outright cancel their action entirely when
performing a destructive action on a dirty project. As of this commit, I
have identified two of such scenarios:
1) closing the project,
2) closing the window with save-on-quit turned off.
If I missed any, do let me know. But for now, maybe now I can finally
start digging into those issues that sent me down this rabbit hole...
See issue #561.
The problem appears to be a due to a combination of factors, such as:
- Python does not automatically convert an empty/blank variable to the
integer zero (0)
- Default goal value is empty/blank for a new Text (scene)
- Asynchronous events can occur such that the change in the Outline
pane of a new Text (scene) goal from empty/blank to a value is not
saved to the data model prior to the update event in the Editor pane
accessing the model value for the word count progress display.
Steps to Reproduce:
1. Start manuskript and create new project (no template).
2. Select **Outline** pane.
3. Click "Text Plus" icon to create a text (default name "New")
4. Select **Editor** pane.
5. Click on **New** to display empty text.
6. Select **Outline** pane.
7. Double-click the empty area on **New** line under title **Goal**,
type in "300", and press **Enter**.
Note that manuskript crashes with a segmentation fault.
Work around the crash by using the already existing manuskript
function toInt() which handles conversion of empty/blank values to
integer value zero (0).
If an invalid character is inserted into the text, such as a "^L" (ASCII 0x0C)
when copy-pasting from a google document that has a page break in it, a crash
will happen as the character cannot be inserted into XML. This patch removes
those invalid characters from the text so the revisions.xml can be saved.
Fixes#562
The About/Settings/Import/Export/ExportManager windows were all created
in odd places, usually to the left of the main window, which meant outside the
desktop area with little overlap if the main window is maximized. The logic in
centering the window on its parent was wrong. This fixes it.
Describing all the rabbitholes that I and kakaroto have gone through
while debugging this one until dawn can frankly not do enough justice to
the crazy amount of rubberducking that went on while trying to fix this.
This bug would be triggered whenever you had a document open in the
editor and then moved an ancestor object downwards (visually) in the tree.
Or when you simply deleted the ancestor. Depending on the exact method
that caused the opened item to be removed from the internal model, the
exact nature of the bug would vary, which means this commit fixes a few
different bits of code that lead to what appears to be the same bug.
In order of appearance, the bugs that ruined our sleep were:
1) The editor widget was trying to handle the removed item at too late a
stage.
2) The editor widget tried to fix its view after a move by searching for
the new item with the same ID, but in the case of moving an object down
it came across its own old item, ruining the attempt.
3) The editor widget did not properly account for the hierarchical
nature of the model.
Upon fixing these the next day, it was revealed that:
4) The outlineItem.updateWordCount(emit=False) flag is broken. This
function would call setData() in several spots which would still cause
emits to bubble through the system despite emit=False, and we simply got
lucky that it stopped enough of them until now.
This last one was caused by a small mistake in the fixes for the first
three bugs, but it has led to a couple of extra changes to make any
future bug hunts slightly less arduous and frustrating:
a) When calling item.removeChild(c), it now resets the associated parent
and model to mirror item.insertChild(c). This has also led to an extra
check in model.parent() to check for its validity.
b) The outlineItem.updateWordCount(emit=) flag has been removed entirely
and it now emits away with reckless abandon. I have been unable to
reproduce the crashes the code warned about, so I consider this a code
quality fix to prevent mysterious future issues where things sometimes
do not properly update right.
Worthy of note is that the original code clearly showed the intention to
close tabs for items that were removed. Reworking the editor to support
closing a tab is unfortunately way out of scope, so this intention was
left in and the new fix was structured to make it trivial to implement
such a change when the time comes. An existing FIXME regarding unrelated
buggy editor behaviour was left in, too.
Many thanks to Kakaroto for burning the midnight oil with me to get to
the bottom of this. (I learned a lot that night!)
Issues #479, #516 and #559 are fixed by this commit. And maybe some others,
too.
Add support for 6.3.8 which has delete_dictionary_entry and do not use gzipped
pickle. Also give higher priority to symspellpy vs pyspellchecker.
List symspellpy dictionaries by order of cached vs non-cached.
symspellpy 6.3.8 is now the minimum version required and add support for showing
that information to the user.
Also add support for spellcheck libraries that are installed but without dicts.
SymSpell is a great spellchecker which works a lot faster than
pyspellchecker for finding suggestions but is a bit slow at
loading dictionaries (about 15 seconds initially, 2 seconds if
using a cached version).
SymSpell also doesn't come with dictionaries, so the code is currently
using dictionaries from pyspellchecker, so if pyspellchecker isn't
installed, then the user won't see any available dictionaries.
Eventually, would need to have an interface for people to manage
dictionaries for it.
Improves the custom dictionary support by making it more generic
and moving it to the base class. Also makes PyEnchant uses a custom
PWL (Personal Word List) file within manuskript's resources directory
and made pyspellchecker detect available languages automatically.
This modifies the Spellchecker abstraction to add a new dictionary support, with
support for pyspellchecker. It also changes the main UI so that multiple libraries
can be supported and dictionaries provided to the user. The custom dictionary of
pyspellchecker has to be handled manually, and the performance and words of this
library isn't on par with PyEnchant, but at least it works with 64 bits.
Fixes#505
This is in preparation for adding support for additional spellchecking libraries
other than PyEnchant which seems to be unmaintained and does not build in
Windows 64 bit.
Issue #549 was caused because the request and reply object urls are not
guaranteed to be the same. Redirects are the most common cause, but a
malformed URL apparently also qualifies. We now make sure to look at the
original request.
Because the code confused me while I was working on it, I decided to
refactor and document it in order to understand what was going on. I am
glad I did: I found another crashing bug involving the rapid-firing of
tooltip requests, and the processing dict never had its entries removed
either, leading to a (very slow) memory leak over time.
All is good in the world of image tooltips now.
The 'displays' system of panels is now changed into simply a settings system
where settings can be associated to widgets. The new API is :
addWidgetSetting, addSetting and setSettingCallback.
The top panel was reworked to have the settings appear in the order of the
widgets, and the path/title choice was changed into a Title widget with a
"Title: Show Full Path" setting.
Realizing that the show/hide progress was being ignored if we navigate to a
scene without a goal set. Also, if we go fullscreen on a scene without a goal
then navigate to a scene with one, the progress wouldn't get shown. Adding the
"Auto Show/Hide" setting fixes the issue with all use cases.
This makes the fullscreen editor much more powerful in terms of navigating
through chapters and scenes. This should make issue #234 users happy and
fix#444.
Top panel now has left/right arrows to navigate through the scenes. It will
automatically find the next/previous text item and display it, navigating through
the outline tree.
There's also a "New document" icon which will create a new text entry immediately
after the current one and switch to it.
Navigation can also be done using Alt+Page-Up, Alt+Page-Down or Alt+Left and Alt+right
shortcuts (Fixing #444).
There's now also the option between Title or Path for the top panel, if Path is chosen
then the full path of the scene is displayed and clicking on the scene or parent items
opens a menu to quickly switch to the selected chapter/scene. Selecting a folder will
automatically display the first text entry available in that folder.
This includes the auto-hide of each panel as well as the shown/hidden status
of each of the displays. Now that it's consistent, it makes it so much more useful.
This is an experimental idea. We can add to a myPanel a list of widgets to show/hide
if the user wants to, via the context menu. This can be very useful for a user who
wants to disable auto-hide for the bottom panel but remove the theme selector which
can be useless to have open permanently.
This is the first step in fixing #234. Would need the auto-hide and the displays
configurations to be saved in settings though before it can become usable.
If manuskript is launched with its last argument set to "--console" an
interactive console opens up to help debug the application.
The IPython, qtconsole and matplotlib libraries must be installed for it
to work and they won't get imported unless the argument is passed to the app.
Color scheme was inverted if foreground was transparent, bug and fix provided
by @tildagail [1] and the text on the left side panel was ignoring text color
settings, making it unreadable in dark themes, as reported by @worstje in #527
[1] https://github.com/olivierkes/manuskript/issues/527#issuecomment-469578130
While tackling issue #529, I stumbled across the odd behaviour that
re-compressing the archive with 7-Zip broke what should be a valid
Manuskript project.
After investigation it turned out that the code that loads the texts
sensibly expects there to only be files tracked in the files dictionary.
It is completely valid for a zip file to contain entries describing the
contained directories. The logical fix is to simply avoid adding these
directory entries to our files dictionary in the first place.
The Cork background and fullscreen theme backgrounds images can now be added
by using the "+" icon from the combobox. Once a file is added, the combobox
is repopulated and the new image is selected.
Note on line 871, in updateThemeBackground, there was a bug where it was using
self.cmbCorkImage instead of self.cmbThemeBackgroundImage
Fixes#399
If you right click once on the fullscreen panel and the context menu pop up
then you right click again somewhere else on the panel *while the previous
context menu is still visible* then it will cause a crash with :
"Windows fatal exception: access violation"
It seems to be caused by a crash in the QT event loop, trying to delete the
existing QMenu within an event handler.
In the properties view, the context menu on the title line would be black
making its content unreadable. Same in the filter line of the "Set Custom icon"
window on the outline's context menu.
Several crashes were encountered opening files on operating systems
that do not default to UTF-8 encoding, such as Windows. In each case
the project file appears to have become corrupted. Because the only
reports to date have been on Windows, attempt to fix by specifying
utf-8 encoding for all text file open methods.
See issues #331, #470, and #502.
Windows path to the image has '\' path separator instead of '/' which makes
the stylesheet fail. Background images don't appear and console gets spammed with :
Could not parse stylesheet of object corkView(0x27248eb6900, name = "corkView")
This prevents any child widget from inheriting the same stylesheet,
more specifically, the context menu of the full screen editor will now
appear normal instead of being black text on black background, which made
it unreadable.
Fixes#440
The lambda function will keep a reference to the scrollbar python object preventing it
from getting destroyed when the QScrollbar is destroyed. This causes the underlying
QT widget to be freed while the python object still exists, therefore the timer itself
doesn't get stopped/cleaned, so the timer will get called and cause a crash with :
"RuntimeError: Wrapped C/C++ object of type myScrollbar has been deleted"
To reproduce, press F11 repeatedly while scrolling.
The menu option "View -> Mode -> Snowflake" has not been implemented.
This greyed out / disabled option has caused some confusion for users.
Remove the snowflake menu option to avoid confusion.
Closes issue #419
See also earlier issue #45
Many applications, including Manuskript in the Editor pane, use the
keyboard shortcut 'Ctrl+Backspace' to delete the previous word.
However in the Plot pane Manuskript uses 'Ctrl+Backspace' to delete a
plot resolution step.
Fix this inconsistent behaviour by removing the keyboard shortcuts for
'Ctrl+Enter' and 'Ctrl+Backspace' from the Plot pane.
Closes issue #375
When running pytest on kubuntu 16.04 the following warning was displayed:
$ python3 -m pytest -v
...
./manuskript/load_save/version_1.py:319: \
UserWarning: Duplicate name: 'outline/0-Folder/0-Text-3.md'
zf.writestr(filename, content, compress_type=compression)
The error was tracked down using the following pytest invocation:
$ python3 -m pytest -v -W error::UserWarning
This invocation showed 4 occurrences similar to the following warning:
self = <zipfile.ZipFile filename='/tmp/tmpgs_sjpzr.msk' mode='w'>
zinfo = <[AttributeError("compress_size") raised in repr()] \
ZipInfo object at 0x7f3cc0124588>
def _writecheck(self, zinfo):
"""Check for errors before writing a file to the archive."""
if zinfo.filename in self.NameToInfo:
import warnings
> warnings.warn('Duplicate name: %r' % zinfo.filename, \
stacklevel=3)
E UserWarning: Duplicate name: 'outline/0-Folder/0-Text-3.md'
These warnings arose in the following 4 tests:
- test_references
- test_autoLoad
- test_loadExportWiget
- test_loadImportWiget
The cause of the issue is that in manuskript/tests/conftest.py, the
mainWindow::closeProject() method is called to close the project, but
the project was never loaded. This meant the zip file setting
defaulted to True, when in fact the Acts sample project is not stored
in a single zip project file.
Fix by removing the call to MW.closeProject() before the project is
loaded.
Manuskript started to segmentation fault on import starting with Qt 5.11.
I found the following link and the Qt bug links within to be useful in
my trouble-shooting efforts.
[Qt 5.11] Various Applications Segfault in 'libfm-qt'
https://github.com/lxqt/libfm-qt/issues/164
Closes issue #402
The root cause was a mismatch between plot IDs and plot model rows.
This issue would appear when a plot was deleted such that the plot IDs
did not match the plot model row numbers and different plots had
different importance levels. The problem would not occur if the most
recently added plot was deleted.
The plot ID / plot model row mismatch was introduced with the
following commit:
Fixes: add plot then choose new plot does not set \
importance slider
3569f78928
Closes issue #404
This reverts commit 2fdf8c64bf.
The problem introduced by the commit is that the Editor pane would
incorrectly label book elements with the plural form of a word. For
example "Books 4", "Sections 2", "Chapters 1", or "Scenes 3". Hence
the need to undo the commit.
Closes issue #383.
See issue #281.
When loading a project that has the setting **Save to one single
file** disabled, Manuskript tries to read all directories and files
under the project directory.
Manuskript expects all files to contain valid unicode characters.
However if a file containing non-unicode characters is read then
Manuskript will crash.
The error message displayed on the console is similar to the
following:
----- begin snippet -----
Traceback (most recent call last):
File "/home/gedakc/workspace/manuskript.olivierkes/bin/../manuskript/ui/welcome.py", line 134, in loadRecentFile
self.mw.loadProject(act.data())
File "/home/gedakc/workspace/manuskript.olivierkes/bin/../manuskript/mainWindow.py", line 566, in loadProject
self.loadDatas(project)
File "/home/gedakc/workspace/manuskript.olivierkes/bin/../manuskript/mainWindow.py", line 793, in loadDatas
errors = loadSave.loadProject(project)
File "/home/gedakc/workspace/manuskript.olivierkes/bin/../manuskript/loadSave.py", line 66, in loadProject
v1.loadProject(project, zip=isZip)
File "/home/gedakc/workspace/manuskript.olivierkes/bin/../manuskript/load_save/version_1.py", line 657, in loadProject
files[os.path.join(p, f)] = fo.read()
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 3131: invalid start byte
----- end snippet -----
There are at least two known situations in which files with
non-unicode characters can arise:
A. The project is on Mac OS X and the operating system automatically
creates a .DS_Store file.
B. The project is under git version control and contains a .git
subdirectory.
This enhancement prevents the Manuskript crash on project load by
ignoring all directory and file names that start with a period.
